[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:21 dist-test-slave-ncp9 krb5kdc[10987](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:21 dist-test-slave-ncp9 krb5kdc[10987](info): set up 2 sockets
Aug 09 08:47:21 dist-test-slave-ncp9 krb5kdc[10987](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 08:47:22.564066 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:22 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:22.681586 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:43989
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:43989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:22.770274 11006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:22.770474 11006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:22.770503 11006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:22.771629 11006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:22.771675 11006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:22.771694 11006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:22.771709 11006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:22.771724 11006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:22.773515 11006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:43989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:43989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:22.773880 11006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:22.774216 11006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:22.784715 11014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:22.785063 11016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:22.785277 11012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:22.785771 11006 server_base.cc:1034] running on GCE node
I20260809 08:47:22.786077 11006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:22.786556 11006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:22.787715 11006 hybrid_clock.cc:648] HybridClock initialized: now 1786265242787710 us; error 52 us; skew 500 ppm
Aug 09 08:47:22 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:22.790337 11006 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:22.790983 11006 webserver.cc:460] Webserver started at http://127.10.184.126:33931/ using document root <none> and password file <none>
I20260809 08:47:22.791311 11006 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:22.791363 11006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:22.791508 11006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:22.792968 11006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "37a6c0c21eff46bcbad1659cc67b2391"
format_stamp: "Formatted at 2026-08-09 08:47:22 on dist-test-slave-ncp9"
server_key: "b3964819a53d842c9fc94ff94fec097a"
server_key_iv: "c595d4ee68962c7f501aacf5b64e8711"
server_key_version: "encryptionkey@0"
I20260809 08:47:22.793612 11006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "37a6c0c21eff46bcbad1659cc67b2391"
format_stamp: "Formatted at 2026-08-09 08:47:22 on dist-test-slave-ncp9"
server_key: "b3964819a53d842c9fc94ff94fec097a"
server_key_iv: "c595d4ee68962c7f501aacf5b64e8711"
server_key_version: "encryptionkey@0"
I20260809 08:47:22.795517 11006 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260809 08:47:22.796481 11029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:22.796717 11006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:22.796854 11006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "37a6c0c21eff46bcbad1659cc67b2391"
format_stamp: "Formatted at 2026-08-09 08:47:22 on dist-test-slave-ncp9"
server_key: "b3964819a53d842c9fc94ff94fec097a"
server_key_iv: "c595d4ee68962c7f501aacf5b64e8711"
server_key_version: "encryptionkey@0"
I20260809 08:47:22.796948 11006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:22.827790 11006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:22.829746 11006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:22.829895 11006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:22.835898 11006 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:43989
I20260809 08:47:22.835968 11124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:43989 every 8 connection(s)
I20260809 08:47:22.836565 11006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:22.838799 11126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:22.842561 11126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: Bootstrap starting.
I20260809 08:47:22.843624 11126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:22.843971 11126 log.cc:824] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:22.844831 11126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: No bootstrap required, opened a new log
I20260809 08:47:22.846859 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11006
I20260809 08:47:22.846974 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:22.846823 11126 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } }
I20260809 08:47:22.847114 11126 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:22.847229 11126 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37a6c0c21eff46bcbad1659cc67b2391, State: Initialized, Role: FOLLOWER
I20260809 08:47:22.847465 11126 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } }
I20260809 08:47:22.847617 11126 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:22.847662 11126 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:22.847714 11126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:22.847800 10977 external_mini_cluster.cc:1428] Setting key 99bc62338f17ae06b5e365d365c62350
I20260809 08:47:22.848393 11126 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } }
I20260809 08:47:22.848555 11126 leader_election.cc:302] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37a6c0c21eff46bcbad1659cc67b2391; no voters:
I20260809 08:47:22.848852 11126 leader_election.cc:288] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:22.848903 11130 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:22.849356 11126 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:22.850833 11130 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [term 1 LEADER]: Becoming Leader. State: Replica: 37a6c0c21eff46bcbad1659cc67b2391, State: Running, Role: LEADER
I20260809 08:47:22.851029 11130 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } }
I20260809 08:47:22.851598 11153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37a6c0c21eff46bcbad1659cc67b2391. Latest consensus state: current_term: 1 leader_uuid: "37a6c0c21eff46bcbad1659cc67b2391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } } }
I20260809 08:47:22.851781 11132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37a6c0c21eff46bcbad1659cc67b2391" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37a6c0c21eff46bcbad1659cc67b2391" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43989 } } }
I20260809 08:47:22.851702 11153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:22.851878 11132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391 [sys.catalog]: This master's current role is: LEADER
W20260809 08:47:22.852941 11157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:22.852994 11157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:22.853114 11158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:22.853385 11158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:22.855170 11158 catalog_manager.cc:1347] Generated new cluster ID: 587d4808bead413d8761b9a06bec31b7
I20260809 08:47:22.855224 11158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:22.859822 11158 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:22.860352 11158 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 08:47:22 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:22.864199 11158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37a6c0c21eff46bcbad1659cc67b2391: Generated new TSK 0
I20260809 08:47:22.864351 11158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:22.867645 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:22.849823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57312 (local address 127.10.184.126:43989)
0809 08:47:22.850082 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:22.850093 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:22.850129 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:47:22.852306 (+ 2177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:22.852312 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:22.852355 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:22.852473 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:22.854899 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:22.857056 (+ 2157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:22.858838 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:22.859532 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:22.859702 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:22.863855 (+ 4153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:22.863859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:22.863859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:22.863870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:22.865845 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:22.866233 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:22.866237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:22.866237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:22.866352 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:22.866590 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:22.866591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:22.866591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:22.866860 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:22.867005 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:22.867236 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 08:47:22.867386 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:22.893514 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:22.982585 11166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:22.982843 11166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:22.982878 11166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:22.984510 11166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:22.984577 11166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:22.984616 11166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:22.986876 11166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:22.987187 11166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:22.987504 11166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:22.994581 11173 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:22.994864 11172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:22.995057 11166 server_base.cc:1034] running on GCE node
W20260809 08:47:22.995121 11175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:22.995404 11166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:22.995647 11166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:22.996781 11166 hybrid_clock.cc:648] HybridClock initialized: now 1786265242996748 us; error 50 us; skew 500 ppm
Aug 09 08:47:22 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:23.000223 11166 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:23.000844 11166 webserver.cc:460] Webserver started at http://127.10.184.65:43213/ using document root <none> and password file <none>
I20260809 08:47:23.001139 11166 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:23.001209 11166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:23.001355 11166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:23.002956 11166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "3cb5674e1a1843ce98a96c107aca03c4"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "4221c17fb2085c5303b94313b867ad1d"
server_key_iv: "3a229d52285a8ee46b1780a0c22697c0"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.003432 11166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "3cb5674e1a1843ce98a96c107aca03c4"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "4221c17fb2085c5303b94313b867ad1d"
server_key_iv: "3a229d52285a8ee46b1780a0c22697c0"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.005647 11166 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 08:47:23.007761 11184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.007963 11166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:23.008057 11166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "3cb5674e1a1843ce98a96c107aca03c4"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "4221c17fb2085c5303b94313b867ad1d"
server_key_iv: "3a229d52285a8ee46b1780a0c22697c0"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.008133 11166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:23.022578 11166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:23.025307 11166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:23.025445 11166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:23.025723 11166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:23.026095 11166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:23.026189 11166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.026237 11166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:23.026260 11166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.039983 11166 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:38111
I20260809 08:47:23.040016 11351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:38111 every 8 connection(s)
I20260809 08:47:23.040446 11166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:23.041049 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11166
I20260809 08:47:23.041142 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:23.041296 10977 external_mini_cluster.cc:1428] Setting key 680beb559822767929936939924d8737
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265242, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
I20260809 08:47:23.053160 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.041609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:57947 (local address 127.10.184.126:43989)
0809 08:47:23.041738 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:23.041740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:23.041975 (+ 235us) server_negotiation.cc:407] Connection header received
0809 08:47:23.042840 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:23.042842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:23.042854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:23.042882 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:23.044431 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.045256 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.046716 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.047035 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.047081 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.050002 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:23.050003 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:23.050004 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:23.050011 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:23.051691 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.052141 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.052141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.052142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.052188 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.052483 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.052484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.052484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.052592 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:23.052650 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:23.053022 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 08:47:23.053095 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260809 08:47:23.053455 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.041615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43989 (local address 127.10.184.65:57947)
0809 08:47:23.041831 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:23.041878 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:23.042723 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:23.043056 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:23.043059 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:23.044022 (+ 963us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:23.044316 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.044323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.045366 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.045366 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.046621 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.046623 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.047186 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.047187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.047331 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.047956 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:23.047968 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:23.049889 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:23.051770 (+ 1881us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.051778 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.051780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.052072 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.052240 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.052240 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.052241 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.052370 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.052666 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:23.052674 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:23.052841 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 08:47:23.053007 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 08:47:23.053181 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":137,"threads_started":1}
I20260809 08:47:23.054055 11353 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43989
I20260809 08:47:23.054152 11353 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:23.054339 11353 heartbeater.cc:511] Master 127.10.184.126:43989 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
I20260809 08:47:23.055202 11064 ts_manager.cc:194] Registered new tserver with Master: 3cb5674e1a1843ce98a96c107aca03c4 (127.10.184.65:38111)
I20260809 08:47:23.056133 11064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:57947
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:23.069049 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:23.168694 11359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:23.168876 11359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:23.168920 11359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:23.170024 11359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:23.170059 11359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:23.170083 11359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:23.171443 11359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:23.171625 11359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:23.171801 11359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:23.176740 11368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:23.176774 11370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:23.176753 11373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:23.176942 11359 server_base.cc:1034] running on GCE node
I20260809 08:47:23.177209 11359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:23.177426 11359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:23.178555 11359 hybrid_clock.cc:648] HybridClock initialized: now 1786265243178547 us; error 27 us; skew 500 ppm
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:23.180963 11359 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:23.181515 11359 webserver.cc:460] Webserver started at http://127.10.184.66:41563/ using document root <none> and password file <none>
I20260809 08:47:23.181730 11359 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:23.181777 11359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:23.181887 11359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:23.183229 11359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "ae803d80e44a4da7971f0e592b3d74cc"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "88a4c4ba918cf930b6883de0f55b2012"
server_key_iv: "2928572520fa2771fe21728a681db797"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.183617 11359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "ae803d80e44a4da7971f0e592b3d74cc"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "88a4c4ba918cf930b6883de0f55b2012"
server_key_iv: "2928572520fa2771fe21728a681db797"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.185160 11359 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:23.185818 11384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.185963 11359 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:23.186048 11359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "ae803d80e44a4da7971f0e592b3d74cc"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "88a4c4ba918cf930b6883de0f55b2012"
server_key_iv: "2928572520fa2771fe21728a681db797"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.186105 11359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:23.194238 11359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:23.195993 11359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:23.196115 11359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:23.196379 11359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:23.196683 11359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:23.196719 11359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.196746 11359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:23.196772 11359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.210191 11359 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:39739
I20260809 08:47:23.210223 11554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:39739 every 8 connection(s)
I20260809 08:47:23.210503 11359 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:23.213286 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11359
I20260809 08:47:23.213372 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:23.213512 10977 external_mini_cluster.cc:1428] Setting key a28eee90bba6d31a9ca217cadf710a38
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:23.220407 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.211176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:36483 (local address 127.10.184.126:43989)
0809 08:47:23.211229 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:23.211230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:23.211424 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:47:23.211934 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:23.211935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:23.211948 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:23.211984 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:23.213014 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.213547 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.214701 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.214889 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.214941 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.217722 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:23.217724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:23.217724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:23.217730 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:23.219152 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.219524 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.219526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.219526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.219588 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.219786 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.219787 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.219788 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.219923 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:23.219990 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:23.220224 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:47:23.220312 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:47:23.220579 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.211150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43989 (local address 127.10.184.66:36483)
0809 08:47:23.211311 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:23.211344 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:23.211840 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:23.212041 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:23.212045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:23.212692 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:23.212929 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.212936 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.213628 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.213629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.214607 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.214610 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.214961 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.214962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.215151 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.215779 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:23.215789 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:23.217630 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:23.219236 (+ 1606us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.219242 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.219244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.219458 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.219648 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.219648 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.219649 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.219736 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.219987 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:23.219992 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:23.220101 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 08:47:23.220179 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:23.220296 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":97,"threads_started":1}
I20260809 08:47:23.221040 11557 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43989
I20260809 08:47:23.221122 11557 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:23.221282 11557 heartbeater.cc:511] Master 127.10.184.126:43989 requested a full tablet report, sending...
I20260809 08:47:23.221669 11064 ts_manager.cc:194] Registered new tserver with Master: ae803d80e44a4da7971f0e592b3d74cc (127.10.184.66:39739)
I20260809 08:47:23.222039 11064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:36483
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:23.241816 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:23.338562 11567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:23.338778 11567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:23.338814 11567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:23.339991 11567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:23.340034 11567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:23.340060 11567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:23.341591 11567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:23.341794 11567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:23.342001 11567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:23.347476 11579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:23.347486 11574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:23.347589 11575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:23.347755 11567 server_base.cc:1034] running on GCE node
I20260809 08:47:23.347939 11567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:23.348167 11567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:23.349308 11567 hybrid_clock.cc:648] HybridClock initialized: now 1786265243349295 us; error 26 us; skew 500 ppm
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:23.351996 11567 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:23.352463 11567 webserver.cc:460] Webserver started at http://127.10.184.67:43295/ using document root <none> and password file <none>
I20260809 08:47:23.352651 11567 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:23.352694 11567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:23.352804 11567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:23.354048 11567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "b6fff38c813248248ae9cc893685db85"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "5b266317ff888975630cb4dbe91cce4b"
server_key_iv: "00316d063950023ae2bab709bafe0140"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.354486 11567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "b6fff38c813248248ae9cc893685db85"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "5b266317ff888975630cb4dbe91cce4b"
server_key_iv: "00316d063950023ae2bab709bafe0140"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.356050 11567 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:23.356818 11589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.356992 11567 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:23.357074 11567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "b6fff38c813248248ae9cc893685db85"
format_stamp: "Formatted at 2026-08-09 08:47:23 on dist-test-slave-ncp9"
server_key: "5b266317ff888975630cb4dbe91cce4b"
server_key_iv: "00316d063950023ae2bab709bafe0140"
server_key_version: "encryptionkey@0"
I20260809 08:47:23.357138 11567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:23.382710 11567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:23.384840 11567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:23.384996 11567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:23.385456 11567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:23.385826 11567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:23.385864 11567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.385887 11567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:23.385910 11567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:23.397284 11567 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:42077
I20260809 08:47:23.397398 11744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:42077 every 8 connection(s)
I20260809 08:47:23.397689 11567 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:23.406136 11137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.398416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:46251 (local address 127.10.184.126:43989)
0809 08:47:23.398490 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:23.398492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:23.398761 (+ 269us) server_negotiation.cc:407] Connection header received
0809 08:47:23.399262 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:23.399263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:23.399274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:23.399321 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:23.400348 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.400887 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.401840 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.402022 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.402070 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.404011 (+ 1941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:23.404013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:23.404014 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:23.404020 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:23.405122 (+ 1102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.405457 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.405458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.405459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.405503 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.405681 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.405682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.405682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.405775 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:23.405818 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:23.406014 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 08:47:23.406072 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:47:23.406329 11751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.398383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43989 (local address 127.10.184.67:46251)
0809 08:47:23.398644 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:23.398685 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:23.399183 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:23.399420 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:23.399425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:23.400069 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:23.400278 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.400283 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.400979 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.400980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.401775 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:23.401778 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.402076 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.402077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:23.402192 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.402652 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:23.402661 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:23.403932 (+ 1271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:23.405193 (+ 1261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.405199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.405201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.405401 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.405545 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:23.405545 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:23.405545 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:23.405628 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:23.405814 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:23.405817 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:23.405923 (+ 106us) client_negotiation.cc:769] Sending connection context
0809 08:47:23.405996 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:47:23.406108 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":80,"threads_started":1}
I20260809 08:47:23.406754 11748 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43989
I20260809 08:47:23.406836 11748 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:23.406999 11748 heartbeater.cc:511] Master 127.10.184.126:43989 requested a full tablet report, sending...
I20260809 08:47:23.407367 11064 ts_manager.cc:194] Registered new tserver with Master: b6fff38c813248248ae9cc893685db85 (127.10.184.67:42077)
I20260809 08:47:23.407374 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11567
I20260809 08:47:23.407444 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:23.407583 10977 external_mini_cluster.cc:1428] Setting key 710c493dd5a2a35f49269ef1c336e461
I20260809 08:47:23.407771 11064 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:46251
I20260809 08:47:23.408752 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:23.432797 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.425553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56152 (local address 127.10.184.65:38111)
0809 08:47:23.425747 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:23.425751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:23.425875 (+ 124us) server_negotiation.cc:407] Connection header received
0809 08:47:23.425927 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:23.425928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:23.425941 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:23.426001 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:23.426532 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.427237 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.427888 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.428221 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.428273 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.430334 (+ 2061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:23.430341 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:23.430342 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:23.430348 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:23.431982 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.432240 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.432242 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.432242 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.432294 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.432410 (+ 116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.432411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.432411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.432522 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:23.432629 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:23.432665 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:47:23.432724 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":89,"threads_started":1}
W20260809 08:47:23.433239 11221 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56152
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[10987](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265243, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:23.456842 11759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:23.450802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56154 (local address 127.10.184.65:38111)
0809 08:47:23.450858 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:23.450860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:23.450930 (+ 70us) server_negotiation.cc:407] Connection header received
0809 08:47:23.450974 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:23.450975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:23.450985 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:23.451030 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:23.451503 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.452047 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.452703 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:23.452949 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:23.453002 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:23.454844 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:23.454846 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:23.454846 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:23.454852 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:23.456060 (+ 1208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.456297 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.456297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.456298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.456343 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:23.456455 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:23.456455 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:23.456455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:23.456529 (+ 74us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:23.456655 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:23.456695 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 08:47:23.456754 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:47:23.457350 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11166
I20260809 08:47:23.469303 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11359
I20260809 08:47:23.483477 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11567
I20260809 08:47:23.492261 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11006
2026-08-09T08:47:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2108 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[11772](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[11772](info): set up 2 sockets
Aug 09 08:47:23 dist-test-slave-ncp9 krb5kdc[11772](info): commencing operation
krb5kdc: starting...
W20260809 08:47:24.610323 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:24 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:24.719697 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39379
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39379
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:24.798048 11791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:24.798237 11791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:24.798264 11791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:24.799360 11791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:24.799400 11791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:24.799419 11791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:24.799435 11791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:24.799450 11791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:24.800915 11791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39379
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39379
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:24.801151 11791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:24.801345 11791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:24.806051 11801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:24.806097 11798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:24.806082 11791 server_base.cc:1034] running on GCE node
W20260809 08:47:24.806280 11799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:24.806519 11791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:24.806795 11791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:24.807932 11791 hybrid_clock.cc:648] HybridClock initialized: now 1786265244807919 us; error 35 us; skew 500 ppm
Aug 09 08:47:24 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265244, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:24.810554 11791 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:24.811019 11791 webserver.cc:460] Webserver started at http://127.10.184.126:37231/ using document root <none> and password file <none>
I20260809 08:47:24.811190 11791 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:24.811228 11791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:24.811316 11791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:24.812295 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "a74d657d3fac4669a7e93551b402b8b8"
format_stamp: "Formatted at 2026-08-09 08:47:24 on dist-test-slave-ncp9"
server_key: "0d21f9974bbcf066385d13cbd6514882"
server_key_iv: "497d9d93d1e574592e2541584ceacd97"
server_key_version: "encryptionkey@0"
I20260809 08:47:24.812587 11791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "a74d657d3fac4669a7e93551b402b8b8"
format_stamp: "Formatted at 2026-08-09 08:47:24 on dist-test-slave-ncp9"
server_key: "0d21f9974bbcf066385d13cbd6514882"
server_key_iv: "497d9d93d1e574592e2541584ceacd97"
server_key_version: "encryptionkey@0"
I20260809 08:47:24.813854 11791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:24.814563 11810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:24.814724 11791 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:24.814806 11791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "a74d657d3fac4669a7e93551b402b8b8"
format_stamp: "Formatted at 2026-08-09 08:47:24 on dist-test-slave-ncp9"
server_key: "0d21f9974bbcf066385d13cbd6514882"
server_key_iv: "497d9d93d1e574592e2541584ceacd97"
server_key_version: "encryptionkey@0"
I20260809 08:47:24.814875 11791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:24.838402 11791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:24.839936 11791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:24.840050 11791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:24.844064 11791 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39379
I20260809 08:47:24.844106 11897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39379 every 8 connection(s)
I20260809 08:47:24.844390 11791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:24.845329 11898 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:24.847695 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8: Bootstrap starting.
I20260809 08:47:24.848248 11898 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:24.848544 11898 log.cc:824] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:24.849236 11898 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8: No bootstrap required, opened a new log
I20260809 08:47:24.850754 11898 raft_consensus.cc:348] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } }
I20260809 08:47:24.850885 11898 raft_consensus.cc:374] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:24.850916 11898 raft_consensus.cc:729] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a74d657d3fac4669a7e93551b402b8b8, State: Initialized, Role: FOLLOWER
I20260809 08:47:24.851038 11898 consensus_queue.cc:260] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } }
I20260809 08:47:24.851099 11898 raft_consensus.cc:388] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:24.851126 11898 raft_consensus.cc:482] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:24.851181 11898 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:24.851652 11898 raft_consensus.cc:504] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } }
I20260809 08:47:24.851751 11898 leader_election.cc:302] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a74d657d3fac4669a7e93551b402b8b8; no voters:
I20260809 08:47:24.851899 11898 leader_election.cc:288] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:24.851939 11903 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:24.852079 11903 raft_consensus.cc:686] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [term 1 LEADER]: Becoming Leader. State: Replica: a74d657d3fac4669a7e93551b402b8b8, State: Running, Role: LEADER
I20260809 08:47:24.852131 11898 sys_catalog.cc:564] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:24.852191 11903 consensus_queue.cc:237] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } }
I20260809 08:47:24.852563 11907 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a74d657d3fac4669a7e93551b402b8b8. Latest consensus state: current_term: 1 leader_uuid: "a74d657d3fac4669a7e93551b402b8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } } }
I20260809 08:47:24.852645 11907 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:24.852775 11904 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a74d657d3fac4669a7e93551b402b8b8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74d657d3fac4669a7e93551b402b8b8" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39379 } } }
I20260809 08:47:24.852856 11917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:24.852867 11904 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:24.853386 11917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:24.853632 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11791
I20260809 08:47:24.853732 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:24.853873 10977 external_mini_cluster.cc:1428] Setting key 270bd3bd6196da4c127739e1fc7b62a8
I20260809 08:47:24.854938 11917 catalog_manager.cc:1347] Generated new cluster ID: d90b0c96e8254efbb334b9aad00f1205
I20260809 08:47:24.854991 11917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:24 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:24.864722 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:24.854462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56494 (local address 127.10.184.126:39379)
0809 08:47:24.854604 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:24.854613 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:24.854658 (+ 45us) server_negotiation.cc:407] Connection header received
0809 08:47:24.854766 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:24.854773 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:24.854824 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:24.854953 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:24.855645 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:24.856985 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:24.858037 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:24.858639 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:24.858730 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:24.861326 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:24.861330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:24.861331 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:24.861343 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:24.863200 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:24.863541 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:24.863544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:24.863544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:24.863661 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:24.863836 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:24.863837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:24.863837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:24.864125 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:24.864282 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:24.864347 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:47:24.864483 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260809 08:47:24.877879 11917 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:24.878541 11917 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:24.883193 11917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a74d657d3fac4669a7e93551b402b8b8: Generated new TSK 0
I20260809 08:47:24.883347 11917 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:24.916069 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:24.995240 11937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:24.995424 11937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:24.995453 11937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:24.996548 11937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:24.996588 11937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:24.996623 11937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:24.998142 11937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:24.998369 11937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:24.998577 11937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:25.003357 11944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.003391 11945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.003374 11948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:25.003641 11937 server_base.cc:1034] running on GCE node
I20260809 08:47:25.003852 11937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:25.004079 11937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:25.005213 11937 hybrid_clock.cc:648] HybridClock initialized: now 1786265245005206 us; error 30 us; skew 500 ppm
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:25.007529 11937 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:25.008057 11937 webserver.cc:460] Webserver started at http://127.10.184.65:38373/ using document root <none> and password file <none>
I20260809 08:47:25.008267 11937 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:25.008315 11937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:25.008440 11937 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:25.009442 11937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "d8c3bdf090e840068b0f56de5c842bdc"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "f9345ed5a2af78fdd104e157f2630767"
server_key_iv: "424e3bda053a6f00ce403dcb5f76303c"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.009773 11937 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "d8c3bdf090e840068b0f56de5c842bdc"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "f9345ed5a2af78fdd104e157f2630767"
server_key_iv: "424e3bda053a6f00ce403dcb5f76303c"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.011090 11937 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:25.011729 11959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.011868 11937 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:25.011950 11937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "d8c3bdf090e840068b0f56de5c842bdc"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "f9345ed5a2af78fdd104e157f2630767"
server_key_iv: "424e3bda053a6f00ce403dcb5f76303c"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.012005 11937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:25.042088 11937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:25.043861 11937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:25.044005 11937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:25.044221 11937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:25.044523 11937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:25.044559 11937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.044587 11937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:25.044605 11937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.049923 11937 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:36137
I20260809 08:47:25.049963 12120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:36137 every 8 connection(s)
I20260809 08:47:25.050248 11937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:25.051030 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11937
I20260809 08:47:25.051129 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:25.051285 10977 external_mini_cluster.cc:1428] Setting key d31e74ff888552d7fb2ecb7dd8492d4d
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:25.060197 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.050909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:58067 (local address 127.10.184.126:39379)
0809 08:47:25.050965 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.050967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.051150 (+ 183us) server_negotiation.cc:407] Connection header received
0809 08:47:25.052689 (+ 1539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.052690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.052703 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.052749 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.053418 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.053941 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.054915 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.055196 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.055263 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.057622 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.057623 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.057623 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.057630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.058865 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.059325 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.059325 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.059325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.059369 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.059612 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.059612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.059612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.059704 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.059751 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.060064 (+ 313us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.060134 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:47:25.060425 12124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.050876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39379 (local address 127.10.184.65:58067)
0809 08:47:25.051042 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.051075 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.052558 (+ 1483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.052844 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.052847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.053136 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:25.053334 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.053339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.054029 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.054031 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.054835 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.054837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.055297 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.055298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.055412 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.055884 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:25.055892 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:25.057515 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:25.058954 (+ 1439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.058962 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.058964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.059255 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.059421 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.059422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.059422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.059549 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.059757 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:25.059763 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:25.059923 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.060048 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.060215 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
I20260809 08:47:25.060943 12122 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39379
I20260809 08:47:25.061038 12122 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:25.061203 12122 heartbeater.cc:511] Master 127.10.184.126:39379 requested a full tablet report, sending...
I20260809 08:47:25.061715 11833 ts_manager.cc:194] Registered new tserver with Master: d8c3bdf090e840068b0f56de5c842bdc (127.10.184.65:36137)
I20260809 08:47:25.062316 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:58067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:25.075577 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:25.154138 12127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:25.154322 12127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:25.154354 12127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:25.155431 12127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:25.155469 12127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:25.155511 12127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:25.156978 12127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:25.157194 12127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:25.157392 12127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:25.162127 12140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.162144 12136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.162151 12137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:25.162314 12127 server_base.cc:1034] running on GCE node
I20260809 08:47:25.162515 12127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:25.162731 12127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:25.163866 12127 hybrid_clock.cc:648] HybridClock initialized: now 1786265245163854 us; error 28 us; skew 500 ppm
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:25.166179 12127 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:25.166548 12127 webserver.cc:460] Webserver started at http://127.10.184.66:38845/ using document root <none> and password file <none>
I20260809 08:47:25.166744 12127 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:25.166790 12127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:25.166908 12127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:25.167858 12127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "0e1c8bf81d3a458c8f65ffadca596a12"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "6a3a2d50d8c965bd92af0981de47cfd4"
server_key_iv: "8d4f3abc5047f6b52b9245f2119d628d"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.168267 12127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "0e1c8bf81d3a458c8f65ffadca596a12"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "6a3a2d50d8c965bd92af0981de47cfd4"
server_key_iv: "8d4f3abc5047f6b52b9245f2119d628d"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.169589 12127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:25.170213 12152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.170359 12127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:25.170439 12127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "0e1c8bf81d3a458c8f65ffadca596a12"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "6a3a2d50d8c965bd92af0981de47cfd4"
server_key_iv: "8d4f3abc5047f6b52b9245f2119d628d"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.170495 12127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:25.191790 12127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:25.193464 12127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:25.193593 12127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:25.193787 12127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:25.194060 12127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:25.194093 12127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.194121 12127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:25.194145 12127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.199507 12127 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:42777
I20260809 08:47:25.199570 12308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:42777 every 8 connection(s)
I20260809 08:47:25.199847 12127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:25.208104 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.200511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:46383 (local address 127.10.184.126:39379)
0809 08:47:25.200575 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.200577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.200743 (+ 166us) server_negotiation.cc:407] Connection header received
0809 08:47:25.201687 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.201688 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.201701 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.201746 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.202348 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.202862 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.203825 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.204026 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.204080 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.206041 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.206043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.206043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.206050 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.207108 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.207437 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.207437 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.207438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.207483 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.207649 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.207649 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.207650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.207741 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.207785 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.207974 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.208040 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:47:25.208251 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.200448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39379 (local address 127.10.184.66:46383)
0809 08:47:25.200621 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.200661 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.201584 (+ 923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.201825 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.201829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.202067 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:25.202260 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.202265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.202944 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.202944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.203742 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.203745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.204094 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.204095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.204214 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.204599 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:25.204606 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:25.205957 (+ 1351us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:25.207175 (+ 1218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.207181 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.207182 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.207382 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.207525 (+ 143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.207526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.207526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.207609 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.207781 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:25.207783 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:25.207887 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.207959 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.208069 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":86,"threads_started":1}
I20260809 08:47:25.208726 12309 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39379
I20260809 08:47:25.208804 12309 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:25.208997 12309 heartbeater.cc:511] Master 127.10.184.126:39379 requested a full tablet report, sending...
I20260809 08:47:25.209357 11833 ts_manager.cc:194] Registered new tserver with Master: 0e1c8bf81d3a458c8f65ffadca596a12 (127.10.184.66:42777)
I20260809 08:47:25.209702 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:46383
I20260809 08:47:25.209759 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12127
I20260809 08:47:25.209847 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:25.210016 10977 external_mini_cluster.cc:1428] Setting key 4010077af2e34f97b88523abf46de5fe
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:25.233078 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:25.312235 12317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:25.312422 12317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:25.312443 12317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:25.313529 12317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:25.313561 12317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:25.313586 12317 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:25.315053 12317 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39379
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:25.315237 12317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:25.315428 12317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:25.320062 12329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.320066 12324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:25.320119 12323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:25.320333 12317 server_base.cc:1034] running on GCE node
I20260809 08:47:25.320556 12317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:25.320749 12317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:25.321883 12317 hybrid_clock.cc:648] HybridClock initialized: now 1786265245321869 us; error 29 us; skew 500 ppm
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:25.324157 12317 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:25.324627 12317 webserver.cc:460] Webserver started at http://127.10.184.67:36523/ using document root <none> and password file <none>
I20260809 08:47:25.324823 12317 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:25.324870 12317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:25.325008 12317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:25.325975 12317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "3e434663e1ec486da343bd8adf462e4b"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "a47e2a643d280956bd35ffe86e53bd2c"
server_key_iv: "cec3498ea62247d60fa465d52f9ff2e2"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.326263 12317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "3e434663e1ec486da343bd8adf462e4b"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "a47e2a643d280956bd35ffe86e53bd2c"
server_key_iv: "cec3498ea62247d60fa465d52f9ff2e2"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.327536 12317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:25.328115 12344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.328256 12317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:25.328331 12317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "3e434663e1ec486da343bd8adf462e4b"
format_stamp: "Formatted at 2026-08-09 08:47:25 on dist-test-slave-ncp9"
server_key: "a47e2a643d280956bd35ffe86e53bd2c"
server_key_iv: "cec3498ea62247d60fa465d52f9ff2e2"
server_key_version: "encryptionkey@0"
I20260809 08:47:25.328393 12317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:25.344165 12317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:25.345893 12317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:25.346014 12317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:25.346223 12317 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:25.346509 12317 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:25.346555 12317 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.346580 12317 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:25.346601 12317 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:25.353632 12317 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:41835
I20260809 08:47:25.353680 12495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:41835 every 8 connection(s)
I20260809 08:47:25.353976 12317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:25.357788 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12317
I20260809 08:47:25.357870 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:25.358022 10977 external_mini_cluster.cc:1428] Setting key 8e54004e1702237c971fd5c244799706
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:25.362731 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.354665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:44733 (local address 127.10.184.126:39379)
0809 08:47:25.354717 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.354720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.354943 (+ 223us) server_negotiation.cc:407] Connection header received
0809 08:47:25.355879 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.355880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.355889 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.355924 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.356507 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.357045 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.358036 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.358295 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.358357 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.360466 (+ 2109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.360467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.360468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.360474 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.361725 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.362053 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.362054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.362054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.362101 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.362280 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.362281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.362281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.362371 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.362415 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.362610 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.362670 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:47:25.362937 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.354646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39379 (local address 127.10.184.67:44733)
0809 08:47:25.354830 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.354872 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.355785 (+ 913us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.355984 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.355987 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.356220 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:25.356448 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.356453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.357112 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.357113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.357965 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.357968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.358386 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.358387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.358534 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.359005 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:25.359014 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:25.360382 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:25.361788 (+ 1406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.361793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.361795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.361986 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.362143 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:25.362143 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:25.362143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:25.362240 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:25.362410 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:25.362413 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:25.362514 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.362592 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.362698 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260809 08:47:25.363343 12496 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39379
I20260809 08:47:25.363426 12496 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:25.363560 12496 heartbeater.cc:511] Master 127.10.184.126:39379 requested a full tablet report, sending...
I20260809 08:47:25.363886 11833 ts_manager.cc:194] Registered new tserver with Master: 3e434663e1ec486da343bd8adf462e4b (127.10.184.67:41835)
I20260809 08:47:25.364246 11833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:44733
I20260809 08:47:25.369709 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:47:25.376811 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.371445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56500 (local address 127.10.184.126:39379)
0809 08:47:25.371480 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.371482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.371700 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:47:25.371754 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.371755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.371765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.371809 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.372384 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.372837 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.373522 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.373703 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.373756 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.374764 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.374766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.374766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.374773 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.375825 (+ 1052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.376121 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.376122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.376122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.376180 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.376338 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.376339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.376339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.376463 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.376551 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.376656 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.376728 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:47:25.379139 11833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56500:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:25.380954 11833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:25.387704 12515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.383679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:36137 (local address 127.0.0.1:57486)
0809 08:47:25.383837 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.383863 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.383925 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.384099 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.384102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.384104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:25.384190 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.384192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.385512 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.385513 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.386564 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.386567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.387422 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.387423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.387511 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.387525 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.387585 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.387633 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260809 08:47:25.387778 12527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.384035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.10.184.67:41835)
0809 08:47:25.384328 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.384332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.384340 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:25.384369 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.384370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.384382 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.384442 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:25.384588 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.385617 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.386780 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.387475 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.387535 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.387586 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.387673 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.387707 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":85,"threads_started":1}
I20260809 08:47:25.388172 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.383783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.10.184.65:36137)
0809 08:47:25.383964 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.383969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.383979 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:25.384015 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.384017 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.384028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.384090 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:25.384243 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.385400 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.386683 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.387266 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.387331 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.387389 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.388043 (+ 654us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.388097 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":82,"threads_started":1}
I20260809 08:47:25.388340 12519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.383700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42777 (local address 127.0.0.1:58172)
0809 08:47:25.384190 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.384200 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.384516 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.384810 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.384811 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.384813 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:25.384972 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.384975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.386207 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.386208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.387243 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.387246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.388136 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.388137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.388196 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.388199 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.388247 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.388282 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":57,"threads_started":1}
I20260809 08:47:25.387815 12511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.383883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:41835 (local address 127.0.0.1:36546)
0809 08:47:25.384222 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.384229 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.384272 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.384449 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.384450 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.384452 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:25.384505 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.384507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.385746 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.385748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.386688 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.386691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.387549 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.387550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.387608 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.387612 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.387659 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.387700 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":8,"thread_start_us":39,"threads_started":1}
I20260809 08:47:25.388392 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.383994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.10.184.66:42777)
0809 08:47:25.384182 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.384186 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.384257 (+ 71us) server_negotiation.cc:407] Connection header received
0809 08:47:25.384634 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.384637 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.384652 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.384718 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:25.385038 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.385907 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.387328 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.388067 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.388133 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.388185 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.388280 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.388324 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":76,"threads_started":1}
I20260809 08:47:25.389026 12034 tablet_service.cc:1467] Processing CreateTablet for tablet a0cbd21fe1944f608d7a88241cdb5bba (DEFAULT_TABLE table=test-table [id=0ecab150e23f45c49a8bd1d8318c06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:25.389113 12216 tablet_service.cc:1467] Processing CreateTablet for tablet a0cbd21fe1944f608d7a88241cdb5bba (DEFAULT_TABLE table=test-table [id=0ecab150e23f45c49a8bd1d8318c06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:25.389382 12034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cbd21fe1944f608d7a88241cdb5bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:25.389422 12216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cbd21fe1944f608d7a88241cdb5bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:25.392462 12528 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Bootstrap starting.
I20260809 08:47:25.392807 12406 tablet_service.cc:1467] Processing CreateTablet for tablet a0cbd21fe1944f608d7a88241cdb5bba (DEFAULT_TABLE table=test-table [id=0ecab150e23f45c49a8bd1d8318c06c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:25.393061 12529 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Bootstrap starting.
I20260809 08:47:25.393132 12406 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0cbd21fe1944f608d7a88241cdb5bba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:25.393362 12528 tablet_bootstrap.cc:654] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:25.394240 12529 tablet_bootstrap.cc:654] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:25.394558 12528 log.cc:824] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:25.395020 12529 log.cc:824] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:25.395517 12528 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: No bootstrap required, opened a new log
I20260809 08:47:25.395593 12528 ts_tablet_manager.cc:1397] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:47:25.395905 12529 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: No bootstrap required, opened a new log
I20260809 08:47:25.395969 12529 ts_tablet_manager.cc:1397] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:47:25.396678 12536 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Bootstrap starting.
I20260809 08:47:25.397637 12536 tablet_bootstrap.cc:654] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:25.398273 12536 log.cc:824] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:25.398460 12528 raft_consensus.cc:348] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.398705 12528 raft_consensus.cc:374] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:25.398759 12528 raft_consensus.cc:729] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8c3bdf090e840068b0f56de5c842bdc, State: Initialized, Role: FOLLOWER
I20260809 08:47:25.398872 12528 consensus_queue.cc:260] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.398746 12529 raft_consensus.cc:348] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.398959 12536 tablet_bootstrap.cc:492] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: No bootstrap required, opened a new log
I20260809 08:47:25.398958 12529 raft_consensus.cc:374] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:25.398999 12529 raft_consensus.cc:729] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e1c8bf81d3a458c8f65ffadca596a12, State: Initialized, Role: FOLLOWER
I20260809 08:47:25.399024 12536 ts_tablet_manager.cc:1397] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:25.399127 12529 consensus_queue.cc:260] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.399255 12528 ts_tablet_manager.cc:1428] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:47:25.399411 12122 heartbeater.cc:503] Master 127.10.184.126:39379 was elected leader, sending a full tablet report...
I20260809 08:47:25.399468 12309 heartbeater.cc:503] Master 127.10.184.126:39379 was elected leader, sending a full tablet report...
I20260809 08:47:25.399857 12529 ts_tablet_manager.cc:1428] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 08:47:25.401813 12536 raft_consensus.cc:348] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.401984 12536 raft_consensus.cc:374] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:25.402025 12536 raft_consensus.cc:729] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e434663e1ec486da343bd8adf462e4b, State: Initialized, Role: FOLLOWER
I20260809 08:47:25.402132 12536 consensus_queue.cc:260] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.402418 12536 ts_tablet_manager.cc:1428] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:47:25.402448 12496 heartbeater.cc:503] Master 127.10.184.126:39379 was elected leader, sending a full tablet report...
W20260809 08:47:25.450333 12311 tablet.cc:2369] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:25.525686 12541 raft_consensus.cc:482] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:25.525826 12541 raft_consensus.cc:504] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.526526 12541 leader_election.cc:288] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8c3bdf090e840068b0f56de5c842bdc (127.10.184.65:36137), 0e1c8bf81d3a458c8f65ffadca596a12 (127.10.184.66:42777)
I20260809 08:47:25.529178 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.526803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59795 (local address 127.10.184.65:36137)
0809 08:47:25.526844 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.526846 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.526856 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:25.526936 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.526937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.526947 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.526992 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:25.527124 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.527595 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.528487 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.528848 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.528932 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.528941 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.529082 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.529120 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:25.529177 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.526745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:36137 (local address 127.10.184.67:59795)
0809 08:47:25.526811 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.526823 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.526873 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.526994 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.526995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.526997 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:25.527067 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.527069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.527661 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.527662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.528406 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.528408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.528937 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.528937 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.529017 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.529021 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.529064 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.529105 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 08:47:25.529595 12061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cbd21fe1944f608d7a88241cdb5bba" candidate_uuid: "3e434663e1ec486da343bd8adf462e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c3bdf090e840068b0f56de5c842bdc" is_pre_election: true
I20260809 08:47:25.529726 12524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.526737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:55003 (local address 127.10.184.66:42777)
0809 08:47:25.526836 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.526838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.527025 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:47:25.527056 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.527057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.527068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.527109 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:25.527304 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.527767 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.528917 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.529326 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.529392 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.529399 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.529628 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.529663 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 08:47:25.529764 12542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.526747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42777 (local address 127.10.184.67:55003)
0809 08:47:25.526944 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:25.526954 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:25.527011 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:25.527121 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:25.527122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:25.527123 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:25.527237 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.527239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.527840 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.527841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.528809 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:25.528812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.529504 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.529506 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:25.529557 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.529562 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:25.529611 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:47:25.529658 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":13,"thread_start_us":138,"threads_started":1}
I20260809 08:47:25.529778 12061 raft_consensus.cc:2454] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e434663e1ec486da343bd8adf462e4b in term 0.
I20260809 08:47:25.530094 12347 leader_election.cc:302] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e434663e1ec486da343bd8adf462e4b, d8c3bdf090e840068b0f56de5c842bdc; no voters:
I20260809 08:47:25.530167 12247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cbd21fe1944f608d7a88241cdb5bba" candidate_uuid: "3e434663e1ec486da343bd8adf462e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" is_pre_election: true
I20260809 08:47:25.530253 12541 raft_consensus.cc:2781] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:25.530320 12541 raft_consensus.cc:482] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:25.530319 12247 raft_consensus.cc:2454] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e434663e1ec486da343bd8adf462e4b in term 0.
I20260809 08:47:25.530350 12541 raft_consensus.cc:3037] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:25.531315 12541 raft_consensus.cc:504] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.531440 12541 leader_election.cc:288] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [CANDIDATE]: Term 1 election: Requested vote from peers d8c3bdf090e840068b0f56de5c842bdc (127.10.184.65:36137), 0e1c8bf81d3a458c8f65ffadca596a12 (127.10.184.66:42777)
I20260809 08:47:25.531577 12247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cbd21fe1944f608d7a88241cdb5bba" candidate_uuid: "3e434663e1ec486da343bd8adf462e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e1c8bf81d3a458c8f65ffadca596a12"
I20260809 08:47:25.531565 12061 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0cbd21fe1944f608d7a88241cdb5bba" candidate_uuid: "3e434663e1ec486da343bd8adf462e4b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8c3bdf090e840068b0f56de5c842bdc"
I20260809 08:47:25.531647 12247 raft_consensus.cc:3037] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:25.531663 12061 raft_consensus.cc:3037] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:25.532512 12061 raft_consensus.cc:2454] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e434663e1ec486da343bd8adf462e4b in term 1.
I20260809 08:47:25.532541 12247 raft_consensus.cc:2454] T a0cbd21fe1944f608d7a88241cdb5bba P 0e1c8bf81d3a458c8f65ffadca596a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e434663e1ec486da343bd8adf462e4b in term 1.
I20260809 08:47:25.532668 12347 leader_election.cc:302] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e434663e1ec486da343bd8adf462e4b, d8c3bdf090e840068b0f56de5c842bdc; no voters:
I20260809 08:47:25.532778 12541 raft_consensus.cc:2781] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:25.532989 12541 raft_consensus.cc:686] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [term 1 LEADER]: Becoming Leader. State: Replica: 3e434663e1ec486da343bd8adf462e4b, State: Running, Role: LEADER
I20260809 08:47:25.533090 12541 consensus_queue.cc:237] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } }
I20260809 08:47:25.534009 11831 catalog_manager.cc:5495] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e434663e1ec486da343bd8adf462e4b (127.10.184.67). New cstate: current_term: 1 leader_uuid: "3e434663e1ec486da343bd8adf462e4b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8c3bdf090e840068b0f56de5c842bdc" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 36137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e1c8bf81d3a458c8f65ffadca596a12" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e434663e1ec486da343bd8adf462e4b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41835 } health_report { overall_health: HEALTHY } } }
W20260809 08:47:25.550953 12123 tablet.cc:2369] T a0cbd21fe1944f608d7a88241cdb5bba P d8c3bdf090e840068b0f56de5c842bdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:25.570309 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.563179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.10.184.65:36137)
0809 08:47:25.563229 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.563231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.563352 (+ 121us) server_negotiation.cc:407] Connection header received
0809 08:47:25.563419 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.563419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.563430 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.563476 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.564062 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.564534 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.565297 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.565484 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.565538 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.567889 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.567899 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.567899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.567906 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.569393 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.569685 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.569688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.569688 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.569746 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.569872 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.569873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.569873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.569993 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.570097 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.570174 (+ 77us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.570244 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:25.592691 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.586980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.10.184.65:36137)
0809 08:47:25.587029 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.587031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.587089 (+ 58us) server_negotiation.cc:407] Connection header received
0809 08:47:25.587156 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.587157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.587166 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.587203 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.587658 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.588100 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.588726 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.588921 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.588969 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.590863 (+ 1894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.590864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.590864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.590870 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.591921 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.592191 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.592192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.592192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.592233 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.592348 (+ 115us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.592349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.592349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.592421 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.592481 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.592578 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.592628 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260809 08:47:25.605038 12497 tablet.cc:2369] T a0cbd21fe1944f608d7a88241cdb5bba P 3e434663e1ec486da343bd8adf462e4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[11772](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265245, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:25.614176 12517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:25.608144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57510 (local address 127.10.184.65:36137)
0809 08:47:25.608195 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:25.608196 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:25.608310 (+ 114us) server_negotiation.cc:407] Connection header received
0809 08:47:25.608354 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:25.608355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:25.608363 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:25.608406 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:25.608878 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.609364 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.610020 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:25.610186 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:25.610227 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:25.612136 (+ 1909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:25.612138 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:25.612138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:25.612143 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:25.613302 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.613594 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.613595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.613595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.613637 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:25.613820 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:25.613820 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:25.613821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:25.613909 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:25.613977 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:25.614049 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:47:25.614117 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 08:47:25.614946 12005 tablet_service.cc:2908] Rejecting scan request for tablet a0cbd21fe1944f608d7a88241cdb5bba: Uninitialized: clean time has not yet been initialized
I20260809 08:47:25.615912 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11937
I20260809 08:47:25.624603 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12127
I20260809 08:47:25.637265 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12317
I20260809 08:47:25.647154 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11791
2026-08-09T08:47:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2155 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[12563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[12563](info): set up 2 sockets
Aug 09 08:47:25 dist-test-slave-ncp9 krb5kdc[12563](info): commencing operation
krb5kdc: starting...
W20260809 08:47:26.774379 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:26 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:26.883669 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44181
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:26.961869 12585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:26.962066 12585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:26.962096 12585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:26.963215 12585 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:26.963256 12585 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:47:26.963274 12585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:26.963290 12585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:26.963305 12585 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:26.963322 12585 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:26.964766 12585 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44181
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:26.965025 12585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:26.965233 12585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:26.969803 12593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:26.969815 12596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:26.969813 12594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:26.969985 12585 server_base.cc:1034] running on GCE node
I20260809 08:47:26.970268 12585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:26.970543 12585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:26.971691 12585 hybrid_clock.cc:648] HybridClock initialized: now 1786265246971675 us; error 29 us; skew 500 ppm
Aug 09 08:47:26 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265246, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:26.976979 12585 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:26.977399 12585 webserver.cc:460] Webserver started at http://127.10.184.126:35033/ using document root <none> and password file <none>
I20260809 08:47:26.977604 12585 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:26.977651 12585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:26.977770 12585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:26.978714 12585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "50545ec7b03f4933a17610ceb6d32b53"
format_stamp: "Formatted at 2026-08-09 08:47:26 on dist-test-slave-ncp9"
server_key: "f5a779ae7e3ce1eb4cecb651b3e6cf27"
server_key_iv: "235543d9b1db58a31b975afca38e673b"
server_key_version: "encryptionkey@0"
I20260809 08:47:26.979032 12585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "50545ec7b03f4933a17610ceb6d32b53"
format_stamp: "Formatted at 2026-08-09 08:47:26 on dist-test-slave-ncp9"
server_key: "f5a779ae7e3ce1eb4cecb651b3e6cf27"
server_key_iv: "235543d9b1db58a31b975afca38e673b"
server_key_version: "encryptionkey@0"
I20260809 08:47:26.980314 12585 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:26.980957 12608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:26.981091 12585 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:26.981184 12585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "50545ec7b03f4933a17610ceb6d32b53"
format_stamp: "Formatted at 2026-08-09 08:47:26 on dist-test-slave-ncp9"
server_key: "f5a779ae7e3ce1eb4cecb651b3e6cf27"
server_key_iv: "235543d9b1db58a31b975afca38e673b"
server_key_version: "encryptionkey@0"
I20260809 08:47:26.981248 12585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:26.993652 12585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:26.995211 12585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:26.995337 12585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:26.999945 12585 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44181
I20260809 08:47:27.000015 12706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44181 every 8 connection(s)
I20260809 08:47:27.000304 12585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:27.000942 12707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.003121 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Bootstrap starting.
I20260809 08:47:27.003732 12707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.003985 12707 log.cc:824] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:27.004642 12707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: No bootstrap required, opened a new log
I20260809 08:47:27.006929 12707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } }
I20260809 08:47:27.007066 12707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.007099 12707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50545ec7b03f4933a17610ceb6d32b53, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.007237 12707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } }
I20260809 08:47:27.007316 12707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:27.007356 12707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:27.007403 12707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.007634 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12585
I20260809 08:47:27.007735 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:27.007894 10977 external_mini_cluster.cc:1428] Setting key df8d53845416cbc166c69c7b99cce50d
I20260809 08:47:27.008073 12707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } }
I20260809 08:47:27.008169 12707 leader_election.cc:302] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50545ec7b03f4933a17610ceb6d32b53; no voters:
I20260809 08:47:27.008330 12707 leader_election.cc:288] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:27.008491 12715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:27.008582 12707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:27.008668 12715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [term 1 LEADER]: Becoming Leader. State: Replica: 50545ec7b03f4933a17610ceb6d32b53, State: Running, Role: LEADER
I20260809 08:47:27.008756 12715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } }
I20260809 08:47:27.009099 12715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50545ec7b03f4933a17610ceb6d32b53. Latest consensus state: current_term: 1 leader_uuid: "50545ec7b03f4933a17610ceb6d32b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } } }
I20260809 08:47:27.009178 12714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50545ec7b03f4933a17610ceb6d32b53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50545ec7b03f4933a17610ceb6d32b53" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44181 } } }
I20260809 08:47:27.009188 12715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:27.009253 12714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:27.009611 12725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:27.010114 12725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:27.011577 12725 catalog_manager.cc:1347] Generated new cluster ID: 220b525a22f94b819c0482fbf0f53c9c
I20260809 08:47:27.011631 12725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265246, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:27.018358 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.008454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.10.184.126:44181)
0809 08:47:27.008625 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.008635 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.008686 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:47:27.008798 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.008806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.008860 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.009017 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.009961 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.011181 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.011949 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.012361 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.012463 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.014639 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.014643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.014643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.014656 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.017010 (+ 2354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.017346 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.017349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.017349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.017443 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.017615 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.017616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.017616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.017834 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.017977 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.018081 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.018209 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":67,"threads_started":1}
I20260809 08:47:27.038177 12725 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:27.038841 12725 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:27.048221 12725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Generated new TSK 0
I20260809 08:47:27.048389 12725 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:27.075829 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44181
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:27.156276 12751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:27.156466 12751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:27.156509 12751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:27.157648 12751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:27.157688 12751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:27.157716 12751 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:27.157739 12751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:27.159164 12751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:44181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:27.159395 12751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:27.159606 12751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:27.164294 12759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:27.164407 12760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:27.164507 12751 server_base.cc:1034] running on GCE node
W20260809 08:47:27.164289 12763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:27.164767 12751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:27.165024 12751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:27.166169 12751 hybrid_clock.cc:648] HybridClock initialized: now 1786265247166153 us; error 31 us; skew 500 ppm
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:27.168617 12751 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:27.169070 12751 webserver.cc:460] Webserver started at http://127.10.184.65:35085/ using document root <none> and password file <none>
I20260809 08:47:27.169288 12751 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:27.169337 12751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:27.169462 12751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:27.170493 12751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "8b3f63ff8056402e93adef407d997239"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "9685c2031b69837a21293783d81a1d1d"
server_key_iv: "f87cd816c101359ba7a8cb46e7fa901b"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.170819 12751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "8b3f63ff8056402e93adef407d997239"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "9685c2031b69837a21293783d81a1d1d"
server_key_iv: "f87cd816c101359ba7a8cb46e7fa901b"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.172075 12751 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:47:27.172729 12771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.172880 12751 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:27.173004 12751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "8b3f63ff8056402e93adef407d997239"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "9685c2031b69837a21293783d81a1d1d"
server_key_iv: "f87cd816c101359ba7a8cb46e7fa901b"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.173070 12751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:27.200614 12751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:27.202765 12751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:27.202901 12751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:27.203383 12751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:27.203420 12751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.203457 12751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:27.203492 12751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:27.210229 12751 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:37847
I20260809 08:47:27.210299 12948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:37847 every 8 connection(s)
I20260809 08:47:27.210614 12751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:27.212592 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.203930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:52489 (local address 127.10.184.126:44181)
0809 08:47:27.203982 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.203984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.204218 (+ 234us) server_negotiation.cc:407] Connection header received
0809 08:47:27.204717 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.204719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.204734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.204804 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.205438 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.206205 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.207190 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.207400 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.207458 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.210127 (+ 2669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.210129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.210129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.210137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.211565 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.211895 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.211896 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.211896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.211942 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.212121 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.212121 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.212122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.212226 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.212271 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.212470 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.212531 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:47:27.212729 12784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.203939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.65:52489)
0809 08:47:27.204095 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.204130 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.204585 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.204877 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.204882 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.205177 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.205357 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.205362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.206301 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.206302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.207097 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.207099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.207481 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.207482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.207586 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.208059 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.208068 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.210049 (+ 1981us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.211633 (+ 1584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.211636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.211637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.211839 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.211996 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.211996 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.211997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.212081 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.212276 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.212279 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.212381 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.212449 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.212562 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260809 08:47:27.213187 12949 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44181
I20260809 08:47:27.213272 12949 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:27.213438 12949 heartbeater.cc:511] Master 127.10.184.126:44181 requested a full tablet report, sending...
I20260809 08:47:27.213909 12630 ts_manager.cc:194] Registered new tserver with Master: 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847)
I20260809 08:47:27.214649 12630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:52489
I20260809 08:47:27.218837 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.214284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:39277 (local address 127.10.184.126:44181)
0809 08:47:27.214319 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.214321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.214420 (+ 99us) server_negotiation.cc:407] Connection header received
0809 08:47:27.214475 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.214476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.214485 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.214525 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.214891 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.215375 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.215994 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.216164 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.216207 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.216869 (+ 662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.216870 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.216870 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.216875 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.218103 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.218358 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.218359 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.218359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.218401 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.218515 (+ 114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.218516 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.218516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.218602 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.218643 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.218730 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.218784 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:27.218839 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.214235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.65:39277)
0809 08:47:27.214358 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.214368 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.214424 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.214538 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.214539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.214702 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.214762 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.214764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.215458 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.215458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.215925 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.215927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.216209 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.216210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.216261 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.216561 (+ 300us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.216565 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.216805 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.218170 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.218170 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.218171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.218307 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.218440 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.218440 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.218440 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.218481 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.218639 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.218640 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.218673 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.218715 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.218781 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":73,"threads_started":1}
I20260809 08:47:27.220669 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12751
I20260809 08:47:27.220757 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:27.220938 10977 external_mini_cluster.cc:1428] Setting key bcafe8293143a9500b031da9f2303737
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:27.243364 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44181
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:27.322422 12965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:27.322607 12965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:27.322638 12965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:27.323753 12965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:27.323792 12965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:27.323818 12965 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:27.323840 12965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:27.325260 12965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:44181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:27.325472 12965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:27.325664 12965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:27.330334 12971 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:27.330335 12977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:27.330379 12975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:27.330382 12965 server_base.cc:1034] running on GCE node
I20260809 08:47:27.330710 12965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:27.330945 12965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:27.332077 12965 hybrid_clock.cc:648] HybridClock initialized: now 1786265247332065 us; error 28 us; skew 500 ppm
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:27.334403 12965 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:27.334848 12965 webserver.cc:460] Webserver started at http://127.10.184.66:40173/ using document root <none> and password file <none>
I20260809 08:47:27.335035 12965 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:27.335081 12965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:27.335207 12965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:27.336131 12965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "6022c2f45a944dad91d5d769432c87b4"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "f62b9ba583344c4af7bb1c542c227b69"
server_key_iv: "36a4e69bc4bd89fab9df596a9f032e98"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.336505 12965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "6022c2f45a944dad91d5d769432c87b4"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "f62b9ba583344c4af7bb1c542c227b69"
server_key_iv: "36a4e69bc4bd89fab9df596a9f032e98"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.338009 12965 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:27.338627 12987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.338778 12965 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:27.338858 12965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "6022c2f45a944dad91d5d769432c87b4"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "f62b9ba583344c4af7bb1c542c227b69"
server_key_iv: "36a4e69bc4bd89fab9df596a9f032e98"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.338915 12965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:27.356770 12965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:27.358366 12965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:27.358486 12965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:27.358917 12965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:27.358958 12965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.358983 12965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:27.358999 12965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:27.365676 12965 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:42757
I20260809 08:47:27.365763 13161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:42757 every 8 connection(s)
I20260809 08:47:27.366022 12965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:27.367457 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12965
I20260809 08:47:27.367563 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:27.367738 10977 external_mini_cluster.cc:1428] Setting key dc01b18fa91e6660dd91367e06085143
I20260809 08:47:27.368057 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.359341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:39219 (local address 127.10.184.126:44181)
0809 08:47:27.359382 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.359384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.359586 (+ 202us) server_negotiation.cc:407] Connection header received
0809 08:47:27.360042 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.360044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.360058 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.360107 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.360918 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.361671 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.362737 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.363035 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.363111 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.365324 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.365325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.365326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.365332 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.366756 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.367137 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.367138 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.367138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.367196 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.367370 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.367371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.367371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.367504 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.367576 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.367885 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.367968 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:27.368203 13000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.359335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.66:39219)
0809 08:47:27.359482 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.359514 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.359948 (+ 434us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.360172 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.360176 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.360494 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.360778 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.360785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.361766 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.361767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.362658 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.362660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.363133 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.363135 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.363277 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.363729 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.363738 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.365235 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.366824 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.366829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.366831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.367070 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.367238 (+ 168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.367238 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.367238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.367322 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.367577 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.367583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.367743 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.367855 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.367999 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":74,"threads_started":1}
I20260809 08:47:27.368640 13162 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44181
I20260809 08:47:27.368721 13162 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:27.368880 13162 heartbeater.cc:511] Master 127.10.184.126:44181 requested a full tablet report, sending...
I20260809 08:47:27.369642 12631 ts_manager.cc:194] Registered new tserver with Master: 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757)
I20260809 08:47:27.370033 12631 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:39219
I20260809 08:47:27.375356 13171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.369558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.66:41989)
0809 08:47:27.369684 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.369694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.369741 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.370331 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.370332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.370540 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.370605 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.370607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.371245 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.371246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.371928 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.371931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.372253 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.372254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.372340 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.372677 (+ 337us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.372682 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.373042 (+ 360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.374372 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.374373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.374374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.374576 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.374724 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.374724 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.374724 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.374790 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.375010 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.375011 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.375056 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.375167 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.375262 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":70,"threads_started":1}
I20260809 08:47:27.375509 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.370159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:41989 (local address 127.10.184.126:44181)
0809 08:47:27.370198 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.370199 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.370207 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:27.370228 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.370229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.370237 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.370287 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.370679 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.371149 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.372005 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.372189 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.372229 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.373116 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.373117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.373117 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.373123 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.374298 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.374641 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.374641 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.374641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.374685 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.374839 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.374839 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.374839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.374956 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.375002 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.375237 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.375395 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:27.391364 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44181
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:27.472010 13175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:27.472195 13175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:27.472224 13175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:27.473383 13175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:27.473423 13175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:27.473450 13175 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:27.473474 13175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:27.474888 13175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:44181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:27.475132 13175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:27.475324 13175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:47:27.480052 13175 server_base.cc:1034] running on GCE node
W20260809 08:47:27.479974 13181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:27.479964 13183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:27.479967 13185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:27.480407 13175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:27.480625 13175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:27.481757 13175 hybrid_clock.cc:648] HybridClock initialized: now 1786265247481743 us; error 31 us; skew 500 ppm
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:27.484032 13175 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:27.484432 13175 webserver.cc:460] Webserver started at http://127.10.184.67:35107/ using document root <none> and password file <none>
I20260809 08:47:27.484659 13175 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:27.484709 13175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:27.484961 13175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:27.485989 13175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "80a34effcda4413cb4c9cd0921779ef2"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "a0bd61c006e744baea1b01ba9ec997d9"
server_key_iv: "29361413c6de42e29dc7399717fbae83"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.486357 13175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "80a34effcda4413cb4c9cd0921779ef2"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "a0bd61c006e744baea1b01ba9ec997d9"
server_key_iv: "29361413c6de42e29dc7399717fbae83"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.487674 13175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:27.488267 13202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.488430 13175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:27.488510 13175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "80a34effcda4413cb4c9cd0921779ef2"
format_stamp: "Formatted at 2026-08-09 08:47:27 on dist-test-slave-ncp9"
server_key: "a0bd61c006e744baea1b01ba9ec997d9"
server_key_iv: "29361413c6de42e29dc7399717fbae83"
server_key_version: "encryptionkey@0"
I20260809 08:47:27.488561 13175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:27.507696 13175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:27.509701 13175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:27.509824 13175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:27.510270 13175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:27.510308 13175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.510327 13175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:27.510340 13175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.516631 13175 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:41959
I20260809 08:47:27.516662 13379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:41959 every 8 connection(s)
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:27.517014 13175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:27.519434 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.510706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:47815 (local address 127.10.184.126:44181)
0809 08:47:27.510752 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.510753 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.510998 (+ 245us) server_negotiation.cc:407] Connection header received
0809 08:47:27.511467 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.511469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.511482 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.511531 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.512273 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.513042 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.514506 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.514721 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.514778 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.517197 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.517198 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.517199 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.517206 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.518369 (+ 1163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.518738 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.518739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.518739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.518790 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.518982 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.518983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.518983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.519077 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.519121 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.519312 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.519373 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:27.519560 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.510742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.67:47815)
0809 08:47:27.510891 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.510924 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.511374 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.511597 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.511601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.511922 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.512189 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.512197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.513141 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.513143 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.514404 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.514408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.514795 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.514796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.514937 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.515429 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.515438 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.517105 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.518456 (+ 1351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.518461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.518463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.518675 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.518846 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.518847 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.518847 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.518939 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.519112 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.519114 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.519222 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.519292 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.519400 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":78,"threads_started":1}
I20260809 08:47:27.520036 13380 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44181
I20260809 08:47:27.520110 13380 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:27.520275 13380 heartbeater.cc:511] Master 127.10.184.126:44181 requested a full tablet report, sending...
I20260809 08:47:27.520610 12630 ts_manager.cc:194] Registered new tserver with Master: 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67:41959)
I20260809 08:47:27.521028 12630 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:47815
I20260809 08:47:27.525563 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13175
I20260809 08:47:27.525648 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:27.525837 10977 external_mini_cluster.cc:1428] Setting key 8a974bea2ccd6e90c0312b90b4e3bdf3
I20260809 08:47:27.525899 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.521087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:37597 (local address 127.10.184.126:44181)
0809 08:47:27.521161 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.521163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.521663 (+ 500us) server_negotiation.cc:407] Connection header received
0809 08:47:27.521684 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.521684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.521693 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.521740 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.521984 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.522437 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.523026 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.523198 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.523237 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.523886 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.523887 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.523887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.523893 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.525057 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.525316 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.525317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.525317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.525360 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.525495 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.525496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.525496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.525575 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.525615 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.525768 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.525832 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:47:27.525915 13390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.521022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.10.184.67:37597)
0809 08:47:27.521244 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.521254 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.521305 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.521749 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.521750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.521888 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:27.521935 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.521936 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.522498 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.522498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.522978 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.522979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.523240 (+ 261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.523240 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.523288 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.523583 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:27.523587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:27.523823 (+ 236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:27.525119 (+ 1296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.525119 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.525120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.525271 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.525393 (+ 122us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:27.525394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:27.525394 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:27.525444 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:27.525624 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:27.525625 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:27.525672 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.525740 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.525831 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":175,"threads_started":1}
I20260809 08:47:27.526718 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:47:27 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:27.547821 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.541968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58058 (local address 127.10.184.126:44181)
0809 08:47:27.542006 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.542007 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.542016 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:27.542058 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.542058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.542066 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.542104 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:27.542628 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.543115 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.543758 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.543924 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.543965 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.545853 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:27.545855 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:27.545855 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:27.545876 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:27.547071 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.547333 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.547333 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.547333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.547377 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:27.547484 (+ 107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:27.547484 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:27.547485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:27.547567 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:27.547630 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.547711 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.547762 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:27.549288 12630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:27.550879 12630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:27.556954 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.552930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37847 (local address 127.0.0.1:47870)
0809 08:47:27.553125 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.553169 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.553238 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.553524 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.553527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.553531 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:27.553596 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.553598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.554589 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.554590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.555740 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.555743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.556670 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.556671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.556748 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.556752 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.556779 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.556854 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":46,"threads_started":1}
I20260809 08:47:27.556965 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.552987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:41959 (local address 127.0.0.1:53552)
0809 08:47:27.553202 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.553211 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.553250 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.553621 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.553622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.553624 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:27.553686 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.553689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.554978 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.554979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.556007 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.556009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.556626 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.556628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.556707 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.556722 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.556783 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.556873 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":31,"thread_start_us":71,"threads_started":1}
I20260809 08:47:27.557344 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.553036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47870 (local address 127.10.184.65:37847)
0809 08:47:27.553373 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.553377 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.553386 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:27.553420 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.553422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.553437 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.553514 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:27.553747 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.554513 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.555834 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.556576 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.557137 (+ 561us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.557216 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.557240 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.557271 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":77,"threads_started":1}
I20260809 08:47:27.557353 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.553057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53552 (local address 127.10.184.67:41959)
0809 08:47:27.553483 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.553488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.553496 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:27.553527 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.553529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.553543 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.553621 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:27.553769 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.554882 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.556077 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.556545 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.556610 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.556680 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.556814 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.556855 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":87,"threads_started":1}
I20260809 08:47:27.557879 12850 tablet_service.cc:1467] Processing CreateTablet for tablet 9c837e1b2150401dadec446c322f2411 (DEFAULT_TABLE table=test-table [id=f72af020146a4d81b3e58ebcce7ee987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.557958 13277 tablet_service.cc:1467] Processing CreateTablet for tablet 9c837e1b2150401dadec446c322f2411 (DEFAULT_TABLE table=test-table [id=f72af020146a4d81b3e58ebcce7ee987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.558218 13418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.553110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42757 (local address 127.0.0.1:56784)
0809 08:47:27.553316 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.553326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.553377 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.553670 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.553671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.553672 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:27.553700 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.553701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.556095 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.556096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.557120 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.557124 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.558008 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.558010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.558069 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.558074 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.558103 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.558142 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":73,"threads_started":1}
I20260809 08:47:27.558282 12850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c837e1b2150401dadec446c322f2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.558302 13277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c837e1b2150401dadec446c322f2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.558596 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.553308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56784 (local address 127.10.184.66:42757)
0809 08:47:27.553513 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.553517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.553527 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:27.553561 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.553563 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.553577 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.553766 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:27.553826 (+ 60us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.554887 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.557204 (+ 2317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.557905 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.558376 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.558440 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.558465 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.558503 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":89,"threads_started":1}
I20260809 08:47:27.559194 13068 tablet_service.cc:1467] Processing CreateTablet for tablet 9c837e1b2150401dadec446c322f2411 (DEFAULT_TABLE table=test-table [id=f72af020146a4d81b3e58ebcce7ee987]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.559530 13068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c837e1b2150401dadec446c322f2411. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.560817 13425 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Bootstrap starting.
I20260809 08:47:27.561697 13426 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Bootstrap starting.
I20260809 08:47:27.561966 13425 tablet_bootstrap.cc:654] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.562386 13426 tablet_bootstrap.cc:654] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.562371 13425 log.cc:824] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:27.562713 13426 log.cc:824] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:27.563088 13429 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Bootstrap starting.
I20260809 08:47:27.563242 13425 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: No bootstrap required, opened a new log
I20260809 08:47:27.563308 13425 ts_tablet_manager.cc:1397] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:47:27.563380 13426 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: No bootstrap required, opened a new log
I20260809 08:47:27.563442 13426 ts_tablet_manager.cc:1397] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:27.563687 13429 tablet_bootstrap.cc:654] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.563943 13429 log.cc:824] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:27.564522 13429 tablet_bootstrap.cc:492] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: No bootstrap required, opened a new log
I20260809 08:47:27.564579 13429 ts_tablet_manager.cc:1397] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:27.565297 13426 raft_consensus.cc:348] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.565476 13426 raft_consensus.cc:374] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.565518 13426 raft_consensus.cc:729] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3f63ff8056402e93adef407d997239, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.565659 13426 consensus_queue.cc:260] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.565595 13425 raft_consensus.cc:348] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.565743 13425 raft_consensus.cc:374] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.565785 13425 raft_consensus.cc:729] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a34effcda4413cb4c9cd0921779ef2, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.565889 13425 consensus_queue.cc:260] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.565934 13426 ts_tablet_manager.cc:1428] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:47:27.566045 12949 heartbeater.cc:503] Master 127.10.184.126:44181 was elected leader, sending a full tablet report...
I20260809 08:47:27.566241 13425 ts_tablet_manager.cc:1428] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:47:27.566380 13380 heartbeater.cc:503] Master 127.10.184.126:44181 was elected leader, sending a full tablet report...
I20260809 08:47:27.566572 13429 raft_consensus.cc:348] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.566730 13429 raft_consensus.cc:374] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.566766 13429 raft_consensus.cc:729] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6022c2f45a944dad91d5d769432c87b4, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.566865 13429 consensus_queue.cc:260] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.567248 13429 ts_tablet_manager.cc:1428] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:47:27.567363 13162 heartbeater.cc:503] Master 127.10.184.126:44181 was elected leader, sending a full tablet report...
I20260809 08:47:27.613293 13435 raft_consensus.cc:482] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:27.613405 13435 raft_consensus.cc:504] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.613770 13435 leader_election.cc:288] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847), 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757)
W20260809 08:47:27.616592 13163 tablet.cc:2369] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:27.617075 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.614016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37847 (local address 127.10.184.67:56545)
0809 08:47:27.614065 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.614076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.614122 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.614246 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.614247 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.614248 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:27.614313 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.614314 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.615160 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.615161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.616237 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.616241 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.616806 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.616807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.616920 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.616923 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.616968 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.617004 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:47:27.617076 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.613991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:56545 (local address 127.10.184.65:37847)
0809 08:47:27.614057 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.614059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.614112 (+ 53us) server_negotiation.cc:407] Connection header received
0809 08:47:27.614167 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.614169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.614179 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.614235 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:27.614374 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.615084 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.616319 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.616740 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.616805 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.616815 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.616982 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.617020 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:47:27.617434 12882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c837e1b2150401dadec446c322f2411" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3f63ff8056402e93adef407d997239" is_pre_election: true
I20260809 08:47:27.617612 12882 raft_consensus.cc:2454] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 0.
I20260809 08:47:27.617790 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.614414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59945 (local address 127.10.184.66:42757)
0809 08:47:27.614483 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.614485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.614494 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:27.614518 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.614519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.614527 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.614585 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:27.614754 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.615348 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.616837 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.617312 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.617364 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.617371 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.617681 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.617729 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:47:27.617821 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.614186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42757 (local address 127.10.184.67:59945)
0809 08:47:27.614414 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:27.614424 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:27.614467 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:27.614587 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:27.614588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:27.614589 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:27.614693 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.614695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.615412 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.615413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.616738 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:27.616742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.617496 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.617497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:27.617601 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.617606 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:27.617667 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:47:27.617704 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":163,"threads_started":1}
I20260809 08:47:27.617897 13206 leader_election.cc:302] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a34effcda4413cb4c9cd0921779ef2, 8b3f63ff8056402e93adef407d997239; no voters:
I20260809 08:47:27.618021 13435 raft_consensus.cc:2781] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:27.618091 13435 raft_consensus.cc:482] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:27.618122 13435 raft_consensus.cc:3037] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.618346 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c837e1b2150401dadec446c322f2411" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4" is_pre_election: true
I20260809 08:47:27.618458 13091 raft_consensus.cc:2454] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 0.
I20260809 08:47:27.619112 13435 raft_consensus.cc:504] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.619233 13435 leader_election.cc:288] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 election: Requested vote from peers 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847), 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757)
I20260809 08:47:27.619393 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c837e1b2150401dadec446c322f2411" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4"
I20260809 08:47:27.619452 13091 raft_consensus.cc:3037] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.620554 13091 raft_consensus.cc:2454] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 1.
I20260809 08:47:27.620712 13206 leader_election.cc:302] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022c2f45a944dad91d5d769432c87b4, 80a34effcda4413cb4c9cd0921779ef2; no voters:
I20260809 08:47:27.620755 12882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c837e1b2150401dadec446c322f2411" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3f63ff8056402e93adef407d997239"
I20260809 08:47:27.620846 12882 raft_consensus.cc:3037] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.621119 13435 raft_consensus.cc:2781] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:27.621296 13435 raft_consensus.cc:686] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 80a34effcda4413cb4c9cd0921779ef2, State: Running, Role: LEADER
I20260809 08:47:27.621466 13435 consensus_queue.cc:237] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } }
I20260809 08:47:27.622045 12882 raft_consensus.cc:2454] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 1.
I20260809 08:47:27.622359 12630 catalog_manager.cc:5495] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "80a34effcda4413cb4c9cd0921779ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:27.632464 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:27.630194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53554 (local address 127.10.184.67:41959)
0809 08:47:27.630241 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:27.630243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:27.630271 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:47:27.630365 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:27.630366 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:27.630380 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:27.630432 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:27.630568 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.631001 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.631758 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:27.631934 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:27.631985 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:27.632186 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:27.632270 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:27.632336 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:27.632381 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:47:27.632400 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:47:27.634702 13091 raft_consensus.cc:1261] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Refusing update from remote peer 80a34effcda4413cb4c9cd0921779ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:27.635003 13435 consensus_queue.cc:1035] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:27.636998 12882 raft_consensus.cc:1261] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Refusing update from remote peer 80a34effcda4413cb4c9cd0921779ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:27.637287 13438 consensus_queue.cc:1035] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:27.637342 13444 mvcc.cc:205] Tried to move back new op lower bound from 7316542454309396480 to 7316542454258315264. Current Snapshot: MvccSnapshot[applied={T|T < 7316542454309396480}]
I20260809 08:47:27.642498 12631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:27.642582 12631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:27.643349 12631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Sending DeleteTablet for 3 replicas of tablet 9c837e1b2150401dadec446c322f2411
I20260809 08:47:27.643766 13277 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c837e1b2150401dadec446c322f2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:27 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:53552
I20260809 08:47:27.643983 13068 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c837e1b2150401dadec446c322f2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:27 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:56784
I20260809 08:47:27.644003 13454 tablet_replica.cc:330] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: stopping tablet replica
I20260809 08:47:27.644208 13454 raft_consensus.cc:2229] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:27.644328 13458 tablet_replica.cc:330] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: stopping tablet replica
I20260809 08:47:27.644400 13458 raft_consensus.cc:2229] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:27.644479 13458 raft_consensus.cc:2258] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:27.644464 13454 raft_consensus.cc:2258] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:27.644826 13458 ts_tablet_manager.cc:1905] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:27.645045 13454 ts_tablet_manager.cc:1905] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:27.646231 12850 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c837e1b2150401dadec446c322f2411 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:27 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:47870
I20260809 08:47:27.646339 13458 ts_tablet_manager.cc:1918] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:27.646411 13458 log.cc:1196] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/9c837e1b2150401dadec446c322f2411
I20260809 08:47:27.646705 13458 ts_tablet_manager.cc:1939] T 9c837e1b2150401dadec446c322f2411 P 6022c2f45a944dad91d5d769432c87b4: Deleting consensus metadata
I20260809 08:47:27.646733 12631 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:27.646883 12631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:27.647104 12611 catalog_manager.cc:4841] TS 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757): tablet 9c837e1b2150401dadec446c322f2411 (table test-table [id=f72af020146a4d81b3e58ebcce7ee987]) successfully deleted
I20260809 08:47:27.647758 13454 ts_tablet_manager.cc:1918] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:27.647894 13454 log.cc:1196] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/9c837e1b2150401dadec446c322f2411
I20260809 08:47:27.648027 13459 tablet_replica.cc:330] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: stopping tablet replica
I20260809 08:47:27.648253 13068 tablet_service.cc:1467] Processing CreateTablet for tablet 89361a4356774687ad51513ee9e9f088 (DEFAULT_TABLE table=test-table [id=830197e67d9c48a68984c41f0db9dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.648253 12850 tablet_service.cc:1467] Processing CreateTablet for tablet 89361a4356774687ad51513ee9e9f088 (DEFAULT_TABLE table=test-table [id=830197e67d9c48a68984c41f0db9dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.648371 12850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89361a4356774687ad51513ee9e9f088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.648391 13068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89361a4356774687ad51513ee9e9f088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.648279 13454 ts_tablet_manager.cc:1939] T 9c837e1b2150401dadec446c322f2411 P 80a34effcda4413cb4c9cd0921779ef2: Deleting consensus metadata
I20260809 08:47:27.648999 13277 tablet_service.cc:1467] Processing CreateTablet for tablet 89361a4356774687ad51513ee9e9f088 (DEFAULT_TABLE table=test-table [id=830197e67d9c48a68984c41f0db9dbb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:27.649106 13277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89361a4356774687ad51513ee9e9f088. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:27.649603 13429 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Bootstrap starting.
I20260809 08:47:27.650297 13429 tablet_bootstrap.cc:654] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.650385 13425 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Bootstrap starting.
I20260809 08:47:27.650470 12609 catalog_manager.cc:4841] TS 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67:41959): tablet 9c837e1b2150401dadec446c322f2411 (table test-table [id=f72af020146a4d81b3e58ebcce7ee987]) successfully deleted
I20260809 08:47:27.650928 13425 tablet_bootstrap.cc:654] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.651124 13429 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: No bootstrap required, opened a new log
I20260809 08:47:27.651193 13429 ts_tablet_manager.cc:1397] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:27.651485 13426 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Bootstrap starting.
I20260809 08:47:27.651453 13429 raft_consensus.cc:348] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.651528 13429 raft_consensus.cc:374] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.651548 13429 raft_consensus.cc:729] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6022c2f45a944dad91d5d769432c87b4, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.651597 13429 consensus_queue.cc:260] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.651688 13429 ts_tablet_manager.cc:1428] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.651993 13425 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: No bootstrap required, opened a new log
I20260809 08:47:27.652037 13425 ts_tablet_manager.cc:1397] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:27.652168 13426 tablet_bootstrap.cc:654] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:27.652176 13425 raft_consensus.cc:348] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.652245 13425 raft_consensus.cc:374] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.652268 13425 raft_consensus.cc:729] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a34effcda4413cb4c9cd0921779ef2, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.652319 13425 consensus_queue.cc:260] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.652406 13425 ts_tablet_manager.cc:1428] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.653085 13459 raft_consensus.cc:2229] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:27.653276 13459 raft_consensus.cc:2258] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:27.653707 13459 ts_tablet_manager.cc:1905] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:27.653751 13426 tablet_bootstrap.cc:492] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: No bootstrap required, opened a new log
I20260809 08:47:27.653792 13426 ts_tablet_manager.cc:1397] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:27.653926 13426 raft_consensus.cc:348] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.653980 13426 raft_consensus.cc:374] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:27.653997 13426 raft_consensus.cc:729] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3f63ff8056402e93adef407d997239, State: Initialized, Role: FOLLOWER
I20260809 08:47:27.654047 13426 consensus_queue.cc:260] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.654134 13426 ts_tablet_manager.cc:1428] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:27.655320 13459 ts_tablet_manager.cc:1918] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:27.655403 13459 log.cc:1196] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/9c837e1b2150401dadec446c322f2411
I20260809 08:47:27.655733 13459 ts_tablet_manager.cc:1939] T 9c837e1b2150401dadec446c322f2411 P 8b3f63ff8056402e93adef407d997239: Deleting consensus metadata
I20260809 08:47:27.656183 12611 catalog_manager.cc:4841] TS 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847): tablet 9c837e1b2150401dadec446c322f2411 (table test-table [id=f72af020146a4d81b3e58ebcce7ee987]) successfully deleted
W20260809 08:47:27.711289 12950 tablet.cc:2369] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:47:27.767542 13382 tablet.cc:2369] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:27.955186 13438 raft_consensus.cc:482] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:27.955293 13438 raft_consensus.cc:504] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.955459 13438 leader_election.cc:288] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847), 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757)
I20260809 08:47:27.955734 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89361a4356774687ad51513ee9e9f088" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4" is_pre_election: true
I20260809 08:47:27.955720 12882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89361a4356774687ad51513ee9e9f088" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3f63ff8056402e93adef407d997239" is_pre_election: true
I20260809 08:47:27.955828 13091 raft_consensus.cc:2454] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 0.
I20260809 08:47:27.955832 12882 raft_consensus.cc:2454] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 0.
I20260809 08:47:27.955976 13206 leader_election.cc:302] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a34effcda4413cb4c9cd0921779ef2, 8b3f63ff8056402e93adef407d997239; no voters:
I20260809 08:47:27.956079 13438 raft_consensus.cc:2781] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:27.956123 13438 raft_consensus.cc:482] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:27.956136 13438 raft_consensus.cc:3037] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.956830 13438 raft_consensus.cc:504] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.956959 13438 leader_election.cc:288] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 election: Requested vote from peers 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847), 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757)
I20260809 08:47:27.957132 12882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89361a4356774687ad51513ee9e9f088" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b3f63ff8056402e93adef407d997239"
I20260809 08:47:27.957206 12882 raft_consensus.cc:3037] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.957178 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89361a4356774687ad51513ee9e9f088" candidate_uuid: "80a34effcda4413cb4c9cd0921779ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4"
I20260809 08:47:27.957245 13091 raft_consensus.cc:3037] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:27.957830 13091 raft_consensus.cc:2454] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 1.
I20260809 08:47:27.957931 12882 raft_consensus.cc:2454] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80a34effcda4413cb4c9cd0921779ef2 in term 1.
I20260809 08:47:27.957991 13206 leader_election.cc:302] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022c2f45a944dad91d5d769432c87b4, 80a34effcda4413cb4c9cd0921779ef2; no voters:
I20260809 08:47:27.958103 13438 raft_consensus.cc:2781] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:27.958159 13438 raft_consensus.cc:686] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 80a34effcda4413cb4c9cd0921779ef2, State: Running, Role: LEADER
I20260809 08:47:27.958212 13438 consensus_queue.cc:237] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:27.958766 12631 catalog_manager.cc:5495] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "80a34effcda4413cb4c9cd0921779ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } health_report { overall_health: HEALTHY } } }
Aug 09 08:47:28 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265246, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:28.023876 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.017583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44181 (local address 127.0.0.1:58072)
0809 08:47:28.017738 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.017748 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.017814 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.017926 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.017927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.018290 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:28.018338 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.018341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.018941 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.018942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.019465 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.019466 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.019771 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.019772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.019817 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.020390 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:28.020395 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:28.021774 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:28.023039 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:28.023046 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:28.023048 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:28.023249 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:28.023391 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:28.023391 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:28.023392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:28.023442 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:28.023609 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:28.023613 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:28.023651 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.023698 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.023783 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":103,"threads_started":1}
I20260809 08:47:28.023921 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.017610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.10.184.126:44181)
0809 08:47:28.017666 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.017668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.017808 (+ 140us) server_negotiation.cc:407] Connection header received
0809 08:47:28.017866 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.017867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.017877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.017920 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:28.018394 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.018886 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.019534 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.019715 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.019763 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.021855 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:28.021856 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:28.021857 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:28.021863 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:28.022971 (+ 1108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.023305 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.023306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.023306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.023350 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.023480 (+ 130us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.023480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.023481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.023569 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:28.023612 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.023717 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.023824 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":24}
I20260809 08:47:28.026081 12631 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:58072:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:47:28.026324 12631 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:28.027588 13068 tablet_service.cc:1467] Processing CreateTablet for tablet 6770d4ed25a44218ac9c5bc5db53284f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a8b5d0cf3e241d7b12144930a9bd184]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:28.027693 12850 tablet_service.cc:1467] Processing CreateTablet for tablet 6770d4ed25a44218ac9c5bc5db53284f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a8b5d0cf3e241d7b12144930a9bd184]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:28.027792 13068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6770d4ed25a44218ac9c5bc5db53284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:28.027844 12850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6770d4ed25a44218ac9c5bc5db53284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:28.028877 13429 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: Bootstrap starting.
I20260809 08:47:28.029120 13426 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: Bootstrap starting.
I20260809 08:47:28.029477 13429 tablet_bootstrap.cc:654] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:28.029594 13426 tablet_bootstrap.cc:654] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:28.027581 13277 tablet_service.cc:1467] Processing CreateTablet for tablet 6770d4ed25a44218ac9c5bc5db53284f (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=9a8b5d0cf3e241d7b12144930a9bd184]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:28.029877 13277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6770d4ed25a44218ac9c5bc5db53284f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:28.030208 13429 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: No bootstrap required, opened a new log
I20260809 08:47:28.030241 13429 ts_tablet_manager.cc:1397] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:28.030449 13429 raft_consensus.cc:348] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.030503 13429 raft_consensus.cc:374] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:28.030514 13429 raft_consensus.cc:729] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6022c2f45a944dad91d5d769432c87b4, State: Initialized, Role: FOLLOWER
I20260809 08:47:28.030550 13429 consensus_queue.cc:260] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.030637 13429 ts_tablet_manager.cc:1428] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:28.030694 13436 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.030761 13436 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.031047 13425 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: Bootstrap starting.
I20260809 08:47:28.031412 13425 tablet_bootstrap.cc:654] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:28.031422 13426 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: No bootstrap required, opened a new log
I20260809 08:47:28.031459 13426 ts_tablet_manager.cc:1397] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:28.031706 13426 raft_consensus.cc:348] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.031774 13426 raft_consensus.cc:374] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:28.031791 13426 raft_consensus.cc:729] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b3f63ff8056402e93adef407d997239, State: Initialized, Role: FOLLOWER
I20260809 08:47:28.031834 13426 consensus_queue.cc:260] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.031924 13426 ts_tablet_manager.cc:1428] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:28.032014 13425 tablet_bootstrap.cc:492] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: No bootstrap required, opened a new log
I20260809 08:47:28.031991 13434 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.032060 13425 ts_tablet_manager.cc:1397] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:28.032094 13434 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.032178 13425 raft_consensus.cc:348] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.032248 13425 raft_consensus.cc:374] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:28.032266 13425 raft_consensus.cc:729] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a34effcda4413cb4c9cd0921779ef2, State: Initialized, Role: FOLLOWER
I20260809 08:47:28.032305 13425 consensus_queue.cc:260] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.032401 13425 ts_tablet_manager.cc:1428] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:28.032413 13438 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.032485 13438 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.073845 13434 raft_consensus.cc:482] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:28.073933 13434 raft_consensus.cc:504] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.074275 13434 leader_election.cc:288] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757), 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67:41959)
I20260809 08:47:28.077222 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.074525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:52027 (local address 127.10.184.67:41959)
0809 08:47:28.074571 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.074573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.074683 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:47:28.074716 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.074717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.074729 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.074779 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:28.074924 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.075415 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.076516 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.076918 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.076972 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.076980 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.077115 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.077153 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:28.077243 13478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.074450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:41959 (local address 127.10.184.65:52027)
0809 08:47:28.074600 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.074610 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.074670 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.074783 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.074784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.074786 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:28.074857 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.074859 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.075484 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.075485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.076423 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.076426 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.076979 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.076980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.077045 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.077049 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.077090 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.077144 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":18,"thread_start_us":77,"threads_started":1}
I20260809 08:47:28.077400 13419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.074505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:38341 (local address 127.10.184.66:42757)
0809 08:47:28.074590 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.074593 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.074651 (+ 58us) server_negotiation.cc:407] Connection header received
0809 08:47:28.074707 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.074708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.074717 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.074760 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:28.074981 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.075521 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.076495 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.076943 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.077013 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.077022 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.077296 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.077340 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 08:47:28.077466 13479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.074435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42757 (local address 127.10.184.65:38341)
0809 08:47:28.074580 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.074592 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.074653 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.074775 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.074777 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.074778 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:28.074913 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.074916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.075598 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.075599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.076421 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.076423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.077024 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.077025 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.077214 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.077219 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.077274 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.077314 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":26,"thread_start_us":96,"threads_started":1}
I20260809 08:47:28.077538 13305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6770d4ed25a44218ac9c5bc5db53284f" candidate_uuid: "8b3f63ff8056402e93adef407d997239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a34effcda4413cb4c9cd0921779ef2" is_pre_election: true
I20260809 08:47:28.077647 13305 raft_consensus.cc:2454] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3f63ff8056402e93adef407d997239 in term 0.
I20260809 08:47:28.077800 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6770d4ed25a44218ac9c5bc5db53284f" candidate_uuid: "8b3f63ff8056402e93adef407d997239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4" is_pre_election: true
I20260809 08:47:28.077889 13091 raft_consensus.cc:2454] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8b3f63ff8056402e93adef407d997239 in term 0.
I20260809 08:47:28.077888 12773 leader_election.cc:302] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80a34effcda4413cb4c9cd0921779ef2, 8b3f63ff8056402e93adef407d997239; no voters:
I20260809 08:47:28.078020 13434 raft_consensus.cc:2781] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:28.078073 13434 raft_consensus.cc:482] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:28.078092 13434 raft_consensus.cc:3037] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:28.078861 13434 raft_consensus.cc:504] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.078994 13434 leader_election.cc:288] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [CANDIDATE]: Term 1 election: Requested vote from peers 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757), 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67:41959)
I20260809 08:47:28.079140 13091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6770d4ed25a44218ac9c5bc5db53284f" candidate_uuid: "8b3f63ff8056402e93adef407d997239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6022c2f45a944dad91d5d769432c87b4"
I20260809 08:47:28.079150 13305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6770d4ed25a44218ac9c5bc5db53284f" candidate_uuid: "8b3f63ff8056402e93adef407d997239" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a34effcda4413cb4c9cd0921779ef2"
I20260809 08:47:28.079214 13305 raft_consensus.cc:3037] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:28.079214 13091 raft_consensus.cc:3037] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:28.079900 13091 raft_consensus.cc:2454] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3f63ff8056402e93adef407d997239 in term 1.
I20260809 08:47:28.080070 12775 leader_election.cc:302] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6022c2f45a944dad91d5d769432c87b4, 8b3f63ff8056402e93adef407d997239; no voters:
I20260809 08:47:28.080165 13434 raft_consensus.cc:2781] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:28.080206 13305 raft_consensus.cc:2454] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8b3f63ff8056402e93adef407d997239 in term 1.
I20260809 08:47:28.080279 13434 raft_consensus.cc:686] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [term 1 LEADER]: Becoming Leader. State: Replica: 8b3f63ff8056402e93adef407d997239, State: Running, Role: LEADER
I20260809 08:47:28.080350 13434 consensus_queue.cc:237] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } }
I20260809 08:47:28.080641 13434 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: TxnStatusTablet state changed. Reason: New leader 8b3f63ff8056402e93adef407d997239. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.080713 13434 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:28.080974 13483 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:47:28.081089 13483 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:47:28.081072 12631 catalog_manager.cc:5495] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8b3f63ff8056402e93adef407d997239 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:28.090875 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.088226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47886 (local address 127.10.184.65:37847)
0809 08:47:28.088270 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.088271 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.088302 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:47:28.088401 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.088402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.088416 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.088473 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:28.088607 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.089103 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.090150 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.090323 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.090365 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.090582 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.090676 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.090728 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.090813 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.090830 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:28.090941 13471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.088150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37847 (local address 127.0.0.1:47886)
0809 08:47:28.088251 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.088263 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.088326 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.088493 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.088494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.088496 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:28.088546 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.088548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.089179 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.089180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.090075 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.090078 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.090389 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.090390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.090474 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.090492 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.090743 (+ 251us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.090751 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.090811 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.090863 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 08:47:28.092163 13091 raft_consensus.cc:1261] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Refusing update from remote peer 8b3f63ff8056402e93adef407d997239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:28.092216 13305 raft_consensus.cc:1261] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Refusing update from remote peer 8b3f63ff8056402e93adef407d997239: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:28.092435 13481 consensus_queue.cc:1035] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:28.092600 13434 consensus_queue.cc:1035] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239 [LEADER]: Connected to new peer: Peer: permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:28.093204 13436 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: TxnStatusTablet state changed. Reason: New leader 8b3f63ff8056402e93adef407d997239. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.093294 13436 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.093255 13438 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: TxnStatusTablet state changed. Reason: New leader 8b3f63ff8056402e93adef407d997239. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.093329 13438 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.093990 13434 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.094070 13434 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:28.094384 13448 mvcc.cc:205] Tried to move back new op lower bound from 7316542456183255040 to 7316542456137539584. Current Snapshot: MvccSnapshot[applied={T|T < 7316542456183255040 or (T in {7316542456183255040})}]
I20260809 08:47:28.094498 13438 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.094568 13438 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 80a34effcda4413cb4c9cd0921779ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.095363 13434 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.095436 13434 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 8b3f63ff8056402e93adef407d997239: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:28.095570 13436 tablet_replica.cc:439] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8b3f63ff8056402e93adef407d997239" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 } } peers { permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 } } peers { permanent_uuid: "80a34effcda4413cb4c9cd0921779ef2" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 41959 } } }
I20260809 08:47:28.095649 13436 tablet_replica.cc:442] T 6770d4ed25a44218ac9c5bc5db53284f P 6022c2f45a944dad91d5d769432c87b4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:28.100351 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.097784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59727 (local address 127.10.184.65:37847)
0809 08:47:28.097825 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.097827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.097966 (+ 139us) server_negotiation.cc:407] Connection header received
0809 08:47:28.098022 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.098022 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.098030 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.098067 (+ 37us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:28.098193 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.098630 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.099715 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.099902 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.099954 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.100140 (+ 186us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.100176 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.100208 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.100294 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.100308 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:28.100425 13497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.097680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37847 (local address 127.10.184.67:59727)
0809 08:47:28.097892 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.097904 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.097968 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.098086 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.098087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.098089 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:28.098136 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.098138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.098697 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.098699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.099638 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.099642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.099980 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.099981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.100050 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.100067 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.100234 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.100236 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.100293 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.100344 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":88,"threads_started":1}
I20260809 08:47:28.102626 12882 raft_consensus.cc:1261] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Refusing update from remote peer 80a34effcda4413cb4c9cd0921779ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:28.102639 13091 raft_consensus.cc:1261] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Refusing update from remote peer 80a34effcda4413cb4c9cd0921779ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:28.102856 13435 consensus_queue.cc:1035] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b3f63ff8056402e93adef407d997239" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:28.102968 13438 consensus_queue.cc:1035] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6022c2f45a944dad91d5d769432c87b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:28.104025 13443 mvcc.cc:205] Tried to move back new op lower bound from 7316542456227094528 to 7316542455637192704. Current Snapshot: MvccSnapshot[applied={T|T < 7316542456227094528}]
I20260809 08:47:28.111858 13510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.108972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:41959 (local address 127.10.184.65:33491)
0809 08:47:28.109119 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:28.109130 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:28.109185 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:28.109295 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:28.109296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:28.109297 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:28.109356 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.109357 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.110268 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.110269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.111006 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:28.111008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.111496 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.111497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:28.111570 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.111585 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.111730 (+ 145us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.111731 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:28.111757 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 08:47:28.111788 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":83,"threads_started":1}
I20260809 08:47:28.111864 13417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.109118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:33491 (local address 127.10.184.67:41959)
0809 08:47:28.109182 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.109184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.109192 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:28.109221 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.109222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.109232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.109293 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:28.109409 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.110194 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.111078 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.111385 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.111453 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.111648 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:28.111679 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:28.111731 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.111774 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.111796 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:47:28.120790 12630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:28.120875 12630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58058:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:28.121438 12630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50545ec7b03f4933a17610ceb6d32b53: Sending DeleteTablet for 3 replicas of tablet 89361a4356774687ad51513ee9e9f088
I20260809 08:47:28.121641 12850 tablet_service.cc:1514] Processing DeleteTablet for tablet 89361a4356774687ad51513ee9e9f088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:28 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:47870
I20260809 08:47:28.121670 13068 tablet_service.cc:1514] Processing DeleteTablet for tablet 89361a4356774687ad51513ee9e9f088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:28 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:56784
I20260809 08:47:28.121668 13277 tablet_service.cc:1514] Processing DeleteTablet for tablet 89361a4356774687ad51513ee9e9f088 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:28 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:53552
I20260809 08:47:28.121727 13459 tablet_replica.cc:330] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: stopping tablet replica
I20260809 08:47:28.121750 13458 tablet_replica.cc:330] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: stopping tablet replica
I20260809 08:47:28.121755 13454 tablet_replica.cc:330] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: stopping tablet replica
I20260809 08:47:28.121769 13459 raft_consensus.cc:2229] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:28.121790 13458 raft_consensus.cc:2229] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:28.121795 13454 raft_consensus.cc:2229] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:28.121800 13459 raft_consensus.cc:2258] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:28.121824 13458 raft_consensus.cc:2258] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:28.121843 13454 raft_consensus.cc:2258] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:28.122128 13454 ts_tablet_manager.cc:1905] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:28.122215 13458 ts_tablet_manager.cc:1905] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:28.123798 13459 ts_tablet_manager.cc:1905] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:28.123817 13454 ts_tablet_manager.cc:1918] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:28.123858 13454 log.cc:1196] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/89361a4356774687ad51513ee9e9f088
I20260809 08:47:28.123890 13458 ts_tablet_manager.cc:1918] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:28.123953 13458 log.cc:1196] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/89361a4356774687ad51513ee9e9f088
I20260809 08:47:28.124059 13454 ts_tablet_manager.cc:1939] T 89361a4356774687ad51513ee9e9f088 P 80a34effcda4413cb4c9cd0921779ef2: Deleting consensus metadata
I20260809 08:47:28.124203 13458 ts_tablet_manager.cc:1939] T 89361a4356774687ad51513ee9e9f088 P 6022c2f45a944dad91d5d769432c87b4: Deleting consensus metadata
I20260809 08:47:28.125021 12609 catalog_manager.cc:4841] TS 80a34effcda4413cb4c9cd0921779ef2 (127.10.184.67:41959): tablet 89361a4356774687ad51513ee9e9f088 (table test-table [id=830197e67d9c48a68984c41f0db9dbb5]) successfully deleted
I20260809 08:47:28.125393 13459 ts_tablet_manager.cc:1918] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:28.125432 13459 log.cc:1196] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/89361a4356774687ad51513ee9e9f088
I20260809 08:47:28.125563 13459 ts_tablet_manager.cc:1939] T 89361a4356774687ad51513ee9e9f088 P 8b3f63ff8056402e93adef407d997239: Deleting consensus metadata
I20260809 08:47:28.125771 12611 catalog_manager.cc:4841] TS 6022c2f45a944dad91d5d769432c87b4 (127.10.184.66:42757): tablet 89361a4356774687ad51513ee9e9f088 (table test-table [id=830197e67d9c48a68984c41f0db9dbb5]) successfully deleted
I20260809 08:47:28.126035 12611 catalog_manager.cc:4841] TS 8b3f63ff8056402e93adef407d997239 (127.10.184.65:37847): tablet 89361a4356774687ad51513ee9e9f088 (table test-table [id=830197e67d9c48a68984c41f0db9dbb5]) successfully deleted
Aug 09 08:47:28 dist-test-slave-ncp9 krb5kdc[12563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265247, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:28.136493 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.124943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47900 (local address 127.10.184.65:37847)
0809 08:47:28.124982 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.124984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.129069 (+ 4085us) server_negotiation.cc:407] Connection header received
0809 08:47:28.129094 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.129095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.129104 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.129151 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:28.129674 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.130160 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.130848 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.131025 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.131083 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.133875 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:28.133884 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:28.133884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:28.133892 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:28.135374 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.135690 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.135693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.135694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.135758 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.135925 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.135925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.135926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.136069 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:28.136208 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.136271 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.136348 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:47:28.136780 12938 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47900
I20260809 08:47:28.143417 12716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:28.137740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.10.184.126:44181)
0809 08:47:28.137790 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:28.137792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:28.137800 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:28.137852 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:28.137853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:28.137863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:28.137907 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:28.138407 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.138896 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.139555 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:28.139760 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:28.139812 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:28.141053 (+ 1241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:28.141055 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:28.141055 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:28.141063 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:28.142445 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.142757 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.142758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.142759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.142813 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:28.142975 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:28.142976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:28.142976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:28.143083 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:28.143166 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:28.143260 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 08:47:28.143331 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260809 08:47:28.143662 12630 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58086
I20260809 08:47:28.144124 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12751
I20260809 08:47:28.156060 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12965
I20260809 08:47:28.166003 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13175
I20260809 08:47:28.176219 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12585
2026-08-09T08:47:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2529 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:28 dist-test-slave-ncp9 krb5kdc[13519](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:28 dist-test-slave-ncp9 krb5kdc[13519](info): set up 2 sockets
Aug 09 08:47:28 dist-test-slave-ncp9 krb5kdc[13519](info): commencing operation
krb5kdc: starting...
W20260809 08:47:29.304418 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:29.412961 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41885
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:29.493172 13534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:29.493360 13534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:29.493391 13534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:29.494486 13534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:29.494529 13534 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:47:29.494546 13534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:29.494563 13534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:29.494578 13534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:29.494594 13534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:29.496052 13534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41885
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:29.496284 13534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:29.496482 13534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:29.501129 13543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.501143 13542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.501274 13546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.501798 13534 server_base.cc:1034] running on GCE node
I20260809 08:47:29.502067 13534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:29.502333 13534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:29.503470 13534 hybrid_clock.cc:648] HybridClock initialized: now 1786265249503457 us; error 28 us; skew 500 ppm
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:29.505879 13534 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:29.506325 13534 webserver.cc:460] Webserver started at http://127.10.184.126:46087/ using document root <none> and password file <none>
I20260809 08:47:29.506520 13534 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:29.506559 13534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:29.506646 13534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:29.507586 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "36411063254048a99175d93ab7304cd3"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "acccc7e4332960284dbdf3c96b934cc5"
server_key_iv: "01c7ecb06506fa2921a8115ce2bac21e"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.507884 13534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "36411063254048a99175d93ab7304cd3"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "acccc7e4332960284dbdf3c96b934cc5"
server_key_iv: "01c7ecb06506fa2921a8115ce2bac21e"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.509039 13534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:29.509668 13558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.509830 13534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:29.509907 13534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "36411063254048a99175d93ab7304cd3"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "acccc7e4332960284dbdf3c96b934cc5"
server_key_iv: "01c7ecb06506fa2921a8115ce2bac21e"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.509959 13534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:29.524145 13534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:29.525772 13534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:29.525874 13534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:29.530097 13534 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:41885
I20260809 08:47:29.530182 13650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:41885 every 8 connection(s)
I20260809 08:47:29.530453 13534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:29.531126 13651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:29.533337 13651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Bootstrap starting.
I20260809 08:47:29.533869 13651 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:29.534097 13651 log.cc:824] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:29.534696 13651 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: No bootstrap required, opened a new log
I20260809 08:47:29.536787 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13534
I20260809 08:47:29.536725 13651 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } }
I20260809 08:47:29.536849 13651 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:29.536870 13651 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36411063254048a99175d93ab7304cd3, State: Initialized, Role: FOLLOWER
I20260809 08:47:29.536913 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:29.537016 13651 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } }
I20260809 08:47:29.537082 10977 external_mini_cluster.cc:1428] Setting key 86e6edce19034a026797d9e341b966ef
I20260809 08:47:29.537101 13651 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:29.537124 13651 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:29.537178 13651 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:29.537832 13651 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } }
I20260809 08:47:29.537926 13651 leader_election.cc:302] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36411063254048a99175d93ab7304cd3; no voters:
I20260809 08:47:29.538079 13651 leader_election.cc:288] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:29.538116 13667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:29.538239 13667 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [term 1 LEADER]: Becoming Leader. State: Replica: 36411063254048a99175d93ab7304cd3, State: Running, Role: LEADER
I20260809 08:47:29.538343 13651 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:29.538321 13667 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } }
I20260809 08:47:29.538681 13667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36411063254048a99175d93ab7304cd3. Latest consensus state: current_term: 1 leader_uuid: "36411063254048a99175d93ab7304cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } } }
I20260809 08:47:29.538761 13667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:29.538980 13674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:29.538944 13669 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36411063254048a99175d93ab7304cd3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36411063254048a99175d93ab7304cd3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41885 } } }
I20260809 08:47:29.539153 13669 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:29.539496 13674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:29.541052 13674 catalog_manager.cc:1347] Generated new cluster ID: f1d818d7ba6f4df1b6f2a9463df2728f
I20260809 08:47:29.541109 13674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:29.549198 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.537577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.10.184.126:41885)
0809 08:47:29.537746 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:29.537753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:29.537780 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:47:29.537868 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:29.537873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:29.537924 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:29.538064 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:29.539002 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.540373 (+ 1371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.541435 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.542027 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.542138 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.544934 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:29.544938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:29.544938 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:29.544951 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:29.547604 (+ 2653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.548003 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.548006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.548006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.548121 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.548326 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.548327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.548327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.548597 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:29.548761 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:29.548833 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 08:47:29.548994 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260809 08:47:29.555750 13674 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:29.556428 13674 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:29.560221 13674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Generated new TSK 0
I20260809 08:47:29.560374 13674 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:29.587651 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41885
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:29.666395 13694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:29.666589 13694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:29.666623 13694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:29.667734 13694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:29.667774 13694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:29.667801 13694 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:47:29.667816 13694 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:29.667845 13694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:29.669286 13694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:29.669504 13694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:29.669711 13694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:29.674582 13700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.674645 13701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.674693 13694 server_base.cc:1034] running on GCE node
W20260809 08:47:29.674583 13703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.675014 13694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:29.675235 13694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:29.676366 13694 hybrid_clock.cc:648] HybridClock initialized: now 1786265249676356 us; error 28 us; skew 500 ppm
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:29.678721 13694 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:29.679098 13694 webserver.cc:460] Webserver started at http://127.10.184.65:43053/ using document root <none> and password file <none>
I20260809 08:47:29.679288 13694 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:29.679330 13694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:29.679443 13694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:29.680387 13694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "5af2b8ea5b444283851ab21fa2535204"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "8c5de8055a10f69a08e92c473a486dee"
server_key_iv: "9be527bec4dcac797c3c109716abcf83"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.680706 13694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "5af2b8ea5b444283851ab21fa2535204"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "8c5de8055a10f69a08e92c473a486dee"
server_key_iv: "9be527bec4dcac797c3c109716abcf83"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.681998 13694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:47:29.682658 13713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.682806 13694 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:29.682883 13694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "5af2b8ea5b444283851ab21fa2535204"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "8c5de8055a10f69a08e92c473a486dee"
server_key_iv: "9be527bec4dcac797c3c109716abcf83"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.682946 13694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:29.693491 13694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:29.695050 13694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:29.695178 13694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:29.695617 13694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:29.695657 13694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.695683 13694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:29.695699 13694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:29.702450 13694 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:34793
I20260809 08:47:29.702540 13886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:34793 every 8 connection(s)
I20260809 08:47:29.702798 13694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:29.705343 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.696107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:54519 (local address 127.10.184.126:41885)
0809 08:47:29.696149 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:29.696151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:29.696358 (+ 207us) server_negotiation.cc:407] Connection header received
0809 08:47:29.696803 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:29.696804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:29.696822 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:29.696913 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:29.697549 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.698132 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.699091 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.699293 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.699347 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.702478 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:29.702480 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:29.702480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:29.702490 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:29.704019 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.704383 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.704384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.704385 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.704447 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.704666 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.704667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.704667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.704784 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:29.704839 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:29.705167 (+ 328us) server_negotiation.cc:299] Negotiation successful
0809 08:47:29.705254 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:29.705480 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.696108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.65:54519)
0809 08:47:29.696250 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:29.696284 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:29.696708 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:29.697013 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:29.697016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:29.697283 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:29.697457 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.697462 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.698218 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.698218 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.699010 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.699013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.699362 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.699363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.699498 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.699996 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:29.700006 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:29.702380 (+ 2374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:29.704100 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.704107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.704109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.704316 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.704501 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.704502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.704503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.704607 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.704839 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:29.704846 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:29.705031 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 08:47:29.705135 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 08:47:29.705282 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":81,"threads_started":1}
I20260809 08:47:29.706048 13887 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41885
I20260809 08:47:29.706128 13887 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:29.706310 13887 heartbeater.cc:511] Master 127.10.184.126:41885 requested a full tablet report, sending...
I20260809 08:47:29.706857 13580 ts_manager.cc:194] Registered new tserver with Master: 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793)
I20260809 08:47:29.707664 13580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:54519
I20260809 08:47:29.712167 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13694
I20260809 08:47:29.712271 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:29.712446 10977 external_mini_cluster.cc:1428] Setting key a677c22f703adcb022c3066d106247c4
I20260809 08:47:29.712513 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.707216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:48341 (local address 127.10.184.126:41885)
0809 08:47:29.707251 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:29.707253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:29.707534 (+ 281us) server_negotiation.cc:407] Connection header received
0809 08:47:29.707604 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:29.707604 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:29.707615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:29.707667 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:29.708140 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.708747 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.709447 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.709652 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.709699 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.710411 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:29.710412 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:29.710412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:29.710418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:29.711613 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.711841 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.711841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.711842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.711882 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.712005 (+ 123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.712006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.712006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.712088 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:29.712130 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:29.712352 (+ 222us) server_negotiation.cc:299] Negotiation successful
0809 08:47:29.712434 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:29.712742 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.707281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.65:48341)
0809 08:47:29.707437 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:29.707469 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:29.707556 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:29.707673 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:29.707674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:29.707993 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:29.708062 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.708065 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.708831 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.708832 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.709388 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.709389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.709702 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.709703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.709751 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.710087 (+ 336us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:29.710091 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:29.710344 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:29.711671 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.711671 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.711671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.711801 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.711921 (+ 120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.711921 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.711921 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.711963 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.712139 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:29.712140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:29.712194 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 08:47:29.712587 (+ 393us) client_negotiation.cc:240] Negotiation successful
0809 08:47:29.712678 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:29.735140 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41885
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:29.813911 13901 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:29.814090 13901 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:29.814111 13901 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:29.815176 13901 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:29.815207 13901 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:29.815225 13901 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:47:29.815238 13901 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:29.815253 13901 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:29.816601 13901 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13901
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:29.816769 13901 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:29.816952 13901 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:29.821643 13912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.821692 13913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.821609 13916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.821815 13901 server_base.cc:1034] running on GCE node
I20260809 08:47:29.822009 13901 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:29.822237 13901 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:29.823386 13901 hybrid_clock.cc:648] HybridClock initialized: now 1786265249823372 us; error 30 us; skew 500 ppm
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:29.825738 13901 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:29.826141 13901 webserver.cc:460] Webserver started at http://127.10.184.66:38935/ using document root <none> and password file <none>
I20260809 08:47:29.826373 13901 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:29.826416 13901 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:29.826504 13901 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:29.827428 13901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "160461e930b24fc9b87ece2a3cec94f7"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "3f97cb881904a1f257af085dbdee3bb5"
server_key_iv: "b856d4636bb767f8bbb530ebb368fff0"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.827714 13901 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "160461e930b24fc9b87ece2a3cec94f7"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "3f97cb881904a1f257af085dbdee3bb5"
server_key_iv: "b856d4636bb767f8bbb530ebb368fff0"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.828969 13901 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:47:29.829627 13929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.829763 13901 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:29.829843 13901 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "160461e930b24fc9b87ece2a3cec94f7"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "3f97cb881904a1f257af085dbdee3bb5"
server_key_iv: "b856d4636bb767f8bbb530ebb368fff0"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.829905 13901 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:29.843451 13901 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:29.845448 13901 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:29.845553 13901 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:29.846024 13901 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:29.846068 13901 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.846087 13901 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:29.846098 13901 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:29.852698 13901 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:41533
I20260809 08:47:29.852792 14107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:41533 every 8 connection(s)
I20260809 08:47:29.853093 13901 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:29.855723 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.846507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:41733 (local address 127.10.184.126:41885)
0809 08:47:29.846573 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:29.846576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:29.846728 (+ 152us) server_negotiation.cc:407] Connection header received
0809 08:47:29.847180 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:29.847181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:29.847196 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:29.847274 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:29.848011 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.848722 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.850250 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.850583 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.850662 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.853166 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:29.853168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:29.853168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:29.853178 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:29.854418 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.854803 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.854804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.854805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.854868 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.855104 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.855105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.855105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.855223 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:29.855287 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:29.855546 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 08:47:29.855632 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:47:29.855911 13940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.846423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.66:41733)
0809 08:47:29.846608 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:29.846647 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:29.847089 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:29.847327 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:29.847330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:29.847655 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:29.847919 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.847925 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.848823 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.848825 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.850118 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.850121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.850692 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.850694 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.850839 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.851278 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:29.851289 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:29.853069 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:29.854504 (+ 1435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.854511 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.854513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.854728 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.854935 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.854936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.854937 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.855042 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.855282 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:29.855289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:29.855418 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 08:47:29.855513 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 08:47:29.855661 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":95,"threads_started":1}
I20260809 08:47:29.856334 14111 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41885
I20260809 08:47:29.856412 14111 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:29.856588 14111 heartbeater.cc:511] Master 127.10.184.126:41885 requested a full tablet report, sending...
I20260809 08:47:29.856995 13580 ts_manager.cc:194] Registered new tserver with Master: 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533)
I20260809 08:47:29.857383 13580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:41733
I20260809 08:47:29.859194 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13901
I20260809 08:47:29.859300 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:29.859479 10977 external_mini_cluster.cc:1428] Setting key 15bde1a2332e8bd87d85227797c4119f
I20260809 08:47:29.862668 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.857444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:35699 (local address 127.10.184.126:41885)
0809 08:47:29.857484 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:29.857486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:29.857591 (+ 105us) server_negotiation.cc:407] Connection header received
0809 08:47:29.857642 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:29.857643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:29.857652 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:29.857705 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:29.858085 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.858542 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.859272 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.859501 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.859557 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.860474 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:29.860475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:29.860475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:29.860481 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:29.861651 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.861941 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.861942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.861942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.861993 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:29.862150 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:29.862151 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:29.862151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:29.862305 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:29.862362 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:29.862513 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 08:47:29.862577 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:29.862691 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:29.857367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.66:35699)
0809 08:47:29.857532 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:29.857543 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:29.857602 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:29.857739 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:29.857740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:29.857941 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:29.858013 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.858016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.858619 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.858620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.859195 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:29.859197 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:29.859589 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:29.859590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:29.859669 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:29.860019 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:29.860023 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:29.860395 (+ 372us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:29.861725 (+ 1330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.861726 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.861726 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.861880 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.862044 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:29.862045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:29.862045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:29.862101 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:29.862365 (+ 264us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:29.862366 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:29.862426 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:47:29.862483 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:47:29.862578 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":58,"threads_started":1}
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:29.884075 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41885
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:29.965806 14131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:29.966002 14131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:29.966027 14131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:29.967065 14131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:29.967103 14131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:29.967124 14131 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 08:47:29.967144 14131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:29.967161 14131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:29.968544 14131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:29.968724 14131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:29.968923 14131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:29.973773 14137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.973843 14131 server_base.cc:1034] running on GCE node
W20260809 08:47:29.973763 14143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:29.973827 14136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:29.974140 14131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:29.974370 14131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:29.975510 14131 hybrid_clock.cc:648] HybridClock initialized: now 1786265249975497 us; error 31 us; skew 500 ppm
Aug 09 08:47:29 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:29.977936 14131 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:29.978325 14131 webserver.cc:460] Webserver started at http://127.10.184.67:43295/ using document root <none> and password file <none>
I20260809 08:47:29.978510 14131 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:29.978556 14131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:29.978684 14131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:29.979631 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "551dc7ef45a34b6983afb55d490837cc"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "34053e5f619ee2808617b0b0f79f74c5"
server_key_iv: "f3818f3fd21e850cbaa0c3bdf0507ba2"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.980026 14131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "551dc7ef45a34b6983afb55d490837cc"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "34053e5f619ee2808617b0b0f79f74c5"
server_key_iv: "f3818f3fd21e850cbaa0c3bdf0507ba2"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.981272 14131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:47:29.981909 14156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.982067 14131 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:29.982148 14131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "551dc7ef45a34b6983afb55d490837cc"
format_stamp: "Formatted at 2026-08-09 08:47:29 on dist-test-slave-ncp9"
server_key: "34053e5f619ee2808617b0b0f79f74c5"
server_key_iv: "f3818f3fd21e850cbaa0c3bdf0507ba2"
server_key_version: "encryptionkey@0"
I20260809 08:47:29.982218 14131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:29.999194 14131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:30.001048 14131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:30.001176 14131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:30.001695 14131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:30.001734 14131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:30.001761 14131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:30.001778 14131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:30 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:30.008548 14131 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:33433
I20260809 08:47:30.008908 14131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:30.008621 14341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:33433 every 8 connection(s)
I20260809 08:47:30.009112 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 14131
I20260809 08:47:30.009224 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:30.009397 10977 external_mini_cluster.cc:1428] Setting key 1e2f14754bb4c8aaac3d9a9addb55eef
I20260809 08:47:30.012014 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.002121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:42973 (local address 127.10.184.126:41885)
0809 08:47:30.002203 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.002205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.002396 (+ 191us) server_negotiation.cc:407] Connection header received
0809 08:47:30.003005 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.003006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.003017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.003061 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:30.003889 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.004622 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.006173 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.006421 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.006524 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.009009 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:30.009011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:30.009011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:30.009020 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:30.010669 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.011059 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.011060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.011060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.011127 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.011355 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.011356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.011356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.011479 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:30.011541 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.011798 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.011911 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:47:30.012120 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.002075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.67:42973)
0809 08:47:30.002260 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.002308 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.002905 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.003142 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.003146 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.003499 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:30.003787 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.003795 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.004735 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.004737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.006067 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.006071 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.006543 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.006544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.006691 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.007127 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:30.007137 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:30.008920 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:30.010758 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.010765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.010767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.010986 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.011186 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.011187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.011188 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.011295 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.011539 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:30.011546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:30.011674 (+ 128us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.011766 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.011917 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":71,"threads_started":1}
I20260809 08:47:30.012511 14342 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41885
I20260809 08:47:30.012593 14342 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:30.012759 14342 heartbeater.cc:511] Master 127.10.184.126:41885 requested a full tablet report, sending...
I20260809 08:47:30.013266 13580 ts_manager.cc:194] Registered new tserver with Master: 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:30.013751 13580 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:42973
I20260809 08:47:30.018460 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.013600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:50591 (local address 127.10.184.126:41885)
0809 08:47:30.013635 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.013637 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.013749 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:47:30.013803 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.013804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.013814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.013856 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:30.014247 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.014706 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.015360 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.015554 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.015605 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.016354 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:30.016355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:30.016355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:30.016362 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:30.017518 (+ 1156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.017806 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.017807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.017807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.017858 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.018015 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.018016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.018016 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.018114 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:30.018167 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.018306 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.018374 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:47:30.018476 14350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.013558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.10.184.67:50591)
0809 08:47:30.013686 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.013696 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.013751 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.013894 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.013895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.014081 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:30.014175 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.014176 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.014790 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.014791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.015291 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.015292 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.015610 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.015611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.015683 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.016004 (+ 321us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:30.016007 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:30.016276 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:30.017591 (+ 1315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.017592 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.017592 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.017746 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.017909 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.017910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.017910 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.017966 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.018174 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:30.018175 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:30.018218 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.018278 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.018376 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":2,"thread_start_us":71,"threads_started":1}
I20260809 08:47:30.020928 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:30 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:47:30 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:30.042598 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.036475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52034 (local address 127.10.184.126:41885)
0809 08:47:30.036527 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.036529 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.036540 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:30.036561 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.036562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.036569 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.036614 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:30.037193 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.037627 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.038312 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.038481 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.038524 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.040622 (+ 2098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:30.040624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:30.040624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:30.040631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:30.041742 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.042062 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.042062 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.042062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.042106 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.042252 (+ 146us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.042253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.042253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.042335 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:30.042401 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.042494 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.042543 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:47:30.044095 13580 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:30.045735 13580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:30.051530 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.047890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33433 (local address 127.0.0.1:41660)
0809 08:47:30.048039 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.048047 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.048084 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.048493 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.048494 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.048497 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.048543 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.048544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049563 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.049564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.050419 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.050423 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051247 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.051324 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.051338 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.051411 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.051459 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":22,"thread_start_us":34,"threads_started":1}
I20260809 08:47:30.051576 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.048040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41660 (local address 127.10.184.67:33433)
0809 08:47:30.048361 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.048364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.048373 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:30.048403 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.048405 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.048418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.048479 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.048599 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049484 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.050527 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051156 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051237 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.051292 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.051438 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.051477 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":85,"threads_started":1}
I20260809 08:47:30.051819 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.047640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41533 (local address 127.0.0.1:51766)
0809 08:47:30.047919 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.047930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.047980 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.048174 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.048177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.048179 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.048238 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.048240 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049471 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.049472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.050689 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.050693 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051591 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.051670 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.051675 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.051711 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.051752 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":68,"threads_started":1}
I20260809 08:47:30.052018 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.047752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48854 (local address 127.10.184.65:34793)
0809 08:47:30.048200 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.048205 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.048214 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:30.048250 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.048252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.048267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.048341 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.048538 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049674 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051030 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051697 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051756 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.051809 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.051890 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.051928 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":75,"threads_started":1}
I20260809 08:47:30.052206 14364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.047638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.0.0.1:48854)
0809 08:47:30.047800 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.047838 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.047902 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.048359 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.048360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.048362 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.048404 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.048406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049754 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.049755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.050947 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.050950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051767 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.051768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.051830 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.051834 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.051878 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.052145 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":52,"threads_started":1}
I20260809 08:47:30.052206 14238 tablet_service.cc:1467] Processing CreateTablet for tablet 51221afb74e240ea9565f253d0c97e14 (DEFAULT_TABLE table=test-table [id=750574d2ea1a46edb1ba07986e673a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.052395 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.047737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51766 (local address 127.10.184.66:41533)
0809 08:47:30.047945 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.047950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.047966 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:47:30.048050 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.048052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.048069 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.048156 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.048304 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.049388 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.050776 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.051475 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.052169 (+ 694us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.052233 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.052261 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.052301 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":81,"threads_started":1}
I20260809 08:47:30.052604 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51221afb74e240ea9565f253d0c97e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.053325 13780 tablet_service.cc:1467] Processing CreateTablet for tablet 51221afb74e240ea9565f253d0c97e14 (DEFAULT_TABLE table=test-table [id=750574d2ea1a46edb1ba07986e673a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.053635 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51221afb74e240ea9565f253d0c97e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.055238 14378 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Bootstrap starting.
I20260809 08:47:30.055680 14005 tablet_service.cc:1467] Processing CreateTablet for tablet 51221afb74e240ea9565f253d0c97e14 (DEFAULT_TABLE table=test-table [id=750574d2ea1a46edb1ba07986e673a60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.056046 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 51221afb74e240ea9565f253d0c97e14. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.056330 14379 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Bootstrap starting.
I20260809 08:47:30.056468 14378 tablet_bootstrap.cc:654] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.056849 14378 log.cc:824] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:30.057180 14379 tablet_bootstrap.cc:654] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.057542 14379 log.cc:824] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:30.057804 14378 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: No bootstrap required, opened a new log
I20260809 08:47:30.057888 14378 ts_tablet_manager.cc:1397] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:47:30.058172 14379 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: No bootstrap required, opened a new log
I20260809 08:47:30.058233 14379 ts_tablet_manager.cc:1397] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:47:30.059723 14379 raft_consensus.cc:348] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.059850 14379 raft_consensus.cc:374] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.059897 14384 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Bootstrap starting.
I20260809 08:47:30.059882 14379 raft_consensus.cc:729] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.059947 14378 raft_consensus.cc:348] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.060106 14378 raft_consensus.cc:374] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.060158 14378 raft_consensus.cc:729] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.060081 14379 consensus_queue.cc:260] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.060281 14378 consensus_queue.cc:260] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.060472 13887 heartbeater.cc:503] Master 127.10.184.126:41885 was elected leader, sending a full tablet report...
I20260809 08:47:30.060531 14384 tablet_bootstrap.cc:654] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.060549 14378 ts_tablet_manager.cc:1428] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:47:30.060662 14342 heartbeater.cc:503] Master 127.10.184.126:41885 was elected leader, sending a full tablet report...
I20260809 08:47:30.060846 14384 log.cc:824] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:30.060463 14379 ts_tablet_manager.cc:1428] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:47:30.061697 14384 tablet_bootstrap.cc:492] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: No bootstrap required, opened a new log
I20260809 08:47:30.061800 14384 ts_tablet_manager.cc:1397] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:47:30.063378 14384 raft_consensus.cc:348] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.063525 14384 raft_consensus.cc:374] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.063575 14384 raft_consensus.cc:729] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.063684 14384 consensus_queue.cc:260] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.063936 14384 ts_tablet_manager.cc:1428] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:47:30.063990 14111 heartbeater.cc:503] Master 127.10.184.126:41885 was elected leader, sending a full tablet report...
W20260809 08:47:30.103673 14112 tablet.cc:2369] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:47:30.203470 13888 tablet.cc:2369] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:47:30.259634 14344 tablet.cc:2369] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:30.313128 14388 raft_consensus.cc:482] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:30.313252 14388 raft_consensus.cc:504] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.313648 14388 leader_election.cc:288] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793)
I20260809 08:47:30.316385 14173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.313830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41533 (local address 127.10.184.67:33551)
0809 08:47:30.313881 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.313893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.313945 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.314070 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.314071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.314073 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.314148 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.314150 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.314833 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.314834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.315594 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.315597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.316129 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.316130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.316213 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.316215 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.316264 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.316300 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":1}
I20260809 08:47:30.316375 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.313887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48675 (local address 127.10.184.65:34793)
0809 08:47:30.313975 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.313978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.314104 (+ 126us) server_negotiation.cc:407] Connection header received
0809 08:47:30.314138 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.314140 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.314148 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.314212 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.314355 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.314814 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.315702 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.316046 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.316104 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.316113 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.316266 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.316303 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 08:47:30.316391 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.313905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:33551 (local address 127.10.184.66:41533)
0809 08:47:30.313956 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.313958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.313969 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:30.313999 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.314000 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.314009 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.314063 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.314236 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.314763 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.315698 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.316059 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.316123 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.316132 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.316282 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.316326 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:30.316408 14394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.313832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.10.184.67:48675)
0809 08:47:30.314042 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.314050 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.314096 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.314221 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.314221 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.314236 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.314297 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.314298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.314889 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.314890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.315608 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.315610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.316124 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.316125 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.316199 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.316202 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.316245 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.316283 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":55,"thread_start_us":90,"threads_started":1}
I20260809 08:47:30.316738 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51221afb74e240ea9565f253d0c97e14" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7" is_pre_election: true
I20260809 08:47:30.316934 14037 raft_consensus.cc:2454] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 0.
I20260809 08:47:30.317071 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51221afb74e240ea9565f253d0c97e14" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204" is_pre_election: true
I20260809 08:47:30.317188 14163 leader_election.cc:302] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 551dc7ef45a34b6983afb55d490837cc; no voters:
I20260809 08:47:30.317220 13817 raft_consensus.cc:2454] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 0.
I20260809 08:47:30.317319 14388 raft_consensus.cc:2781] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:30.317378 14388 raft_consensus.cc:482] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:30.317401 14388 raft_consensus.cc:3037] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.318389 14388 raft_consensus.cc:504] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.318503 14388 leader_election.cc:288] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 election: Requested vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793)
I20260809 08:47:30.318675 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51221afb74e240ea9565f253d0c97e14" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204"
I20260809 08:47:30.318688 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "51221afb74e240ea9565f253d0c97e14" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7"
I20260809 08:47:30.318761 14037 raft_consensus.cc:3037] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.318761 13817 raft_consensus.cc:3037] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.319583 14037 raft_consensus.cc:2454] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 1.
I20260809 08:47:30.319583 13817 raft_consensus.cc:2454] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 1.
I20260809 08:47:30.319767 14158 leader_election.cc:302] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551dc7ef45a34b6983afb55d490837cc, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:30.319844 14388 raft_consensus.cc:2781] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:30.319981 14388 raft_consensus.cc:686] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 1 LEADER]: Becoming Leader. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Running, Role: LEADER
I20260809 08:47:30.320079 14388 consensus_queue.cc:237] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.320909 13580 catalog_manager.cc:5495] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 551dc7ef45a34b6983afb55d490837cc (127.10.184.67). New cstate: current_term: 1 leader_uuid: "551dc7ef45a34b6983afb55d490837cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } health_report { overall_health: HEALTHY } } }
Aug 09 08:47:30 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265249, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:30.351588 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.345517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52044 (local address 127.10.184.126:41885)
0809 08:47:30.345579 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.345581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.345645 (+ 64us) server_negotiation.cc:407] Connection header received
0809 08:47:30.345700 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.345701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.345710 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.345758 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:30.346184 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.346661 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.347326 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.347499 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.347549 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.349476 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:30.349478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:30.349478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:30.349484 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:30.350688 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.351043 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.351044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.351045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.351097 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:30.351233 (+ 136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:30.351234 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:30.351234 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:30.351321 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:30.351363 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.351465 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.351527 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:47:30.351696 14402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.345429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41885 (local address 127.0.0.1:52044)
0809 08:47:30.345576 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.345588 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.345652 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.345768 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.345769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.346062 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:30.346105 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.346107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.346725 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.346726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.347254 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.347256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.347553 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.347553 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.347602 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.348070 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:30.348075 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:30.349402 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:30.350746 (+ 1344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.350750 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.350752 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.350993 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.351146 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:30.351147 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:30.351147 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:30.351200 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:30.351359 (+ 159us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:30.351363 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:30.351406 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.351449 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.351586 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":53,"thread_start_us":77,"threads_started":1}
I20260809 08:47:30.353346 13580 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:52044:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:47:30.353547 13580 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:30.354864 13780 tablet_service.cc:1467] Processing CreateTablet for tablet c111199a72464549a6ebead3ffd72e39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83574ac345bb441883379b8479530756]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:30.354872 14005 tablet_service.cc:1467] Processing CreateTablet for tablet c111199a72464549a6ebead3ffd72e39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83574ac345bb441883379b8479530756]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:30.354882 14238 tablet_service.cc:1467] Processing CreateTablet for tablet c111199a72464549a6ebead3ffd72e39 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83574ac345bb441883379b8479530756]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:30.355027 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c111199a72464549a6ebead3ffd72e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.355027 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c111199a72464549a6ebead3ffd72e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.355029 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c111199a72464549a6ebead3ffd72e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.356232 14378 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: Bootstrap starting.
I20260809 08:47:30.356514 14379 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: Bootstrap starting.
I20260809 08:47:30.356660 14378 tablet_bootstrap.cc:654] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.357055 14379 tablet_bootstrap.cc:654] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.357347 14378 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: No bootstrap required, opened a new log
I20260809 08:47:30.357360 14384 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: Bootstrap starting.
I20260809 08:47:30.357391 14378 ts_tablet_manager.cc:1397] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:30.357532 14378 raft_consensus.cc:348] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.357606 14378 raft_consensus.cc:374] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.357620 14378 raft_consensus.cc:729] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.357657 14378 consensus_queue.cc:260] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.357702 14384 tablet_bootstrap.cc:654] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.357697 14379 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: No bootstrap required, opened a new log
I20260809 08:47:30.357738 14379 ts_tablet_manager.cc:1397] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:30.357764 14378 ts_tablet_manager.cc:1428] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:30.357784 14388 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.357849 14388 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.357892 14379 raft_consensus.cc:348] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.357954 14379 raft_consensus.cc:374] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.357971 14379 raft_consensus.cc:729] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.358013 14379 consensus_queue.cc:260] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.358127 14379 ts_tablet_manager.cc:1428] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:30.358173 14385 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.358268 14385 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.358405 14384 tablet_bootstrap.cc:492] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: No bootstrap required, opened a new log
I20260809 08:47:30.358450 14384 ts_tablet_manager.cc:1397] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:30.358769 14384 raft_consensus.cc:348] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.358837 14384 raft_consensus.cc:374] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.358850 14384 raft_consensus.cc:729] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.358879 14384 consensus_queue.cc:260] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.358959 14384 ts_tablet_manager.cc:1428] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:30.359077 14390 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.359155 14390 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.566639 14385 raft_consensus.cc:482] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:30.566746 14385 raft_consensus.cc:504] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.567067 14385 leader_election.cc:288] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:30.570670 14408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.567263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33433 (local address 127.10.184.65:36527)
0809 08:47:30.567384 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.567396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.567457 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.567610 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.567611 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.567612 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.567722 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.567725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.568598 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.568598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.569928 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.569932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.570474 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570474 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.570522 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.570526 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.570563 (+ 37us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.570594 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":58,"thread_start_us":56,"threads_started":1}
I20260809 08:47:30.570694 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.567327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:36527 (local address 127.10.184.67:33433)
0809 08:47:30.567427 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.567428 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.567441 (+ 13us) server_negotiation.cc:407] Connection header received
0809 08:47:30.567520 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.567521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.567530 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.567597 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.567786 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.568535 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570006 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.570388 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570457 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.570466 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.570583 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.570621 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82,"thread_start_us":44,"threads_started":1}
I20260809 08:47:30.571053 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c111199a72464549a6ebead3ffd72e39" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc" is_pre_election: true
I20260809 08:47:30.571153 14411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.567268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41533 (local address 127.10.184.65:33715)
0809 08:47:30.567384 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.567396 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.567457 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.567608 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.567609 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.567611 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:30.567722 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.567725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.568597 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.568598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.570292 (+ 1694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.570295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.570946 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.571017 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.571021 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.571060 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.571087 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":70,"thread_start_us":59,"threads_started":1}
I20260809 08:47:30.571187 14277 raft_consensus.cc:2454] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 0.
I20260809 08:47:30.571194 14372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.567311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:33715 (local address 127.10.184.66:41533)
0809 08:47:30.567389 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.567391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.567459 (+ 68us) server_negotiation.cc:407] Connection header received
0809 08:47:30.567516 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.567517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.567527 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.567587 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:30.567785 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.568529 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570361 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.570842 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.570907 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.570914 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.571089 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.571135 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 08:47:30.571365 13714 leader_election.cc:302] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551dc7ef45a34b6983afb55d490837cc, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:30.571455 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c111199a72464549a6ebead3ffd72e39" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7" is_pre_election: true
I20260809 08:47:30.571507 14385 raft_consensus.cc:2781] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:30.571540 14037 raft_consensus.cc:2454] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 0.
I20260809 08:47:30.571553 14385 raft_consensus.cc:482] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:30.571574 14385 raft_consensus.cc:3037] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.572321 14385 raft_consensus.cc:504] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.572448 14385 leader_election.cc:288] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 election: Requested vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:30.572577 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c111199a72464549a6ebead3ffd72e39" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7"
I20260809 08:47:30.572589 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c111199a72464549a6ebead3ffd72e39" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc"
I20260809 08:47:30.572644 14037 raft_consensus.cc:3037] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.572649 14277 raft_consensus.cc:3037] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.573262 14037 raft_consensus.cc:2454] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 1.
I20260809 08:47:30.573309 14277 raft_consensus.cc:2454] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 1.
I20260809 08:47:30.573444 13718 leader_election.cc:302] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:30.573570 14385 raft_consensus.cc:2781] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:30.573697 14385 raft_consensus.cc:686] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [term 1 LEADER]: Becoming Leader. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Running, Role: LEADER
I20260809 08:47:30.573808 14385 consensus_queue.cc:237] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.574221 14413 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: TxnStatusTablet state changed. Reason: New leader 5af2b8ea5b444283851ab21fa2535204. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.574308 14413 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:30.574563 14415 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:47:30.574672 14415 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:47:30.574595 13580 catalog_manager.cc:5495] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:30.583266 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.580802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48862 (local address 127.10.184.65:34793)
0809 08:47:30.580857 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.580859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.580865 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:47:30.580909 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.580910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.580928 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.580994 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:30.581132 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.581574 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.582630 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.582795 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.582831 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.583012 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.583095 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.583143 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.583208 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.583222 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:47:30.583325 14402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.580538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.0.0.1:48862)
0809 08:47:30.580624 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.580637 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.580704 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.581019 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.581020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.581022 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:30.581072 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.581075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.581640 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.581641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.582554 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.582558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.582849 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.582850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.582915 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.582933 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.583147 (+ 214us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.583153 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.583205 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.583254 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:47:30.585067 14277 raft_consensus.cc:1261] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Refusing update from remote peer 5af2b8ea5b444283851ab21fa2535204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:30.585069 14037 raft_consensus.cc:1261] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Refusing update from remote peer 5af2b8ea5b444283851ab21fa2535204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:30.585361 14385 consensus_queue.cc:1035] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:30.585484 14413 consensus_queue.cc:1035] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:30.586556 14388 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: TxnStatusTablet state changed. Reason: New leader 5af2b8ea5b444283851ab21fa2535204. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.586651 14388 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.586618 14390 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: TxnStatusTablet state changed. Reason: New leader 5af2b8ea5b444283851ab21fa2535204. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.586710 14390 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.587625 14413 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.587661 14390 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.587713 14413 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:30.587723 14390 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 160461e930b24fc9b87ece2a3cec94f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.587776 14422 mvcc.cc:205] Tried to move back new op lower bound from 7316542466394095616 to 7316542466350841856. Current Snapshot: MvccSnapshot[applied={T|T < 7316542466394095616}]
I20260809 08:47:30.588279 14413 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.588352 14413 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 5af2b8ea5b444283851ab21fa2535204: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:30.588773 14388 tablet_replica.cc:439] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } } }
I20260809 08:47:30.588842 14388 tablet_replica.cc:442] T c111199a72464549a6ebead3ffd72e39 P 551dc7ef45a34b6983afb55d490837cc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:30.589352 14424 mvcc.cc:205] Tried to move back new op lower bound from 7316542466394095616 to 7316542466350841856. Current Snapshot: MvccSnapshot[applied={T|T < 7316542466394095616 or (T in {7316542466394095616})}]
I20260809 08:47:30.592176 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.589780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41672 (local address 127.10.184.67:33433)
0809 08:47:30.589816 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.589818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.589868 (+ 50us) server_negotiation.cc:407] Connection header received
0809 08:47:30.589942 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.589943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.589956 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.590006 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:30.590111 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.590539 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.591582 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.591753 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.591792 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.591937 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.592012 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.592059 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.592109 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.592125 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:30.597007 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.594532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:52941 (local address 127.10.184.65:34793)
0809 08:47:30.594576 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.594578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.594763 (+ 185us) server_negotiation.cc:407] Connection header received
0809 08:47:30.594817 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.594818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.594825 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.594866 (+ 41us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:30.594978 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.595407 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.596435 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.596596 (+ 161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.596633 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.596811 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.596833 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.596863 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.596939 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.596954 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:47:30.597055 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.594454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.10.184.67:52941)
0809 08:47:30.594695 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.594705 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.594765 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.594870 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.594871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.594872 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:30.594926 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.594927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.595464 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.595464 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.596368 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.596371 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.596649 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.596650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.596720 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.596734 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.596866 (+ 132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.596868 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.596931 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.596971 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":162,"threads_started":1}
I20260809 08:47:30.599004 13817 raft_consensus.cc:1261] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Refusing update from remote peer 551dc7ef45a34b6983afb55d490837cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:30.599149 14037 raft_consensus.cc:1261] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Refusing update from remote peer 551dc7ef45a34b6983afb55d490837cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:30.599359 14388 consensus_queue.cc:1035] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:30.599476 14395 consensus_queue.cc:1035] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:30.601069 14423 mvcc.cc:205] Tried to move back new op lower bound from 7316542466451787776 to 7316542465311551488. Current Snapshot: MvccSnapshot[applied={T|T < 7316542466451787776}]
I20260809 08:47:30.607996 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.605340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40427 (local address 127.10.184.67:33433)
0809 08:47:30.605382 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:30.605384 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:30.605487 (+ 103us) server_negotiation.cc:407] Connection header received
0809 08:47:30.605613 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:30.605614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:30.605625 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:30.605685 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:30.605812 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.606523 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.607338 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.607587 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.607640 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.607788 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.607819 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.607861 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:30.607912 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 08:47:30.607932 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:30.608033 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:30.605199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33433 (local address 127.10.184.65:40427)
0809 08:47:30.605430 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:30.605441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:30.605496 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:30.605700 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:30.605701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:30.605703 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:30.605759 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.605760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.606591 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.606591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.607273 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:30.607277 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:30.607652 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:30.607653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:30.607711 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:30.607724 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:30.607871 (+ 147us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:30.607873 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:30.607911 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 08:47:30.607947 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":84,"threads_started":1}
I20260809 08:47:30.617202 13579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:30.617293 13579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:30.617947 13579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Sending DeleteTablet for 3 replicas of tablet 51221afb74e240ea9565f253d0c97e14
I20260809 08:47:30.618348 14238 tablet_service.cc:1514] Processing DeleteTablet for tablet 51221afb74e240ea9565f253d0c97e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:30 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41660
I20260809 08:47:30.618405 13579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:30.618554 13579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:30.618549 13780 tablet_service.cc:1514] Processing DeleteTablet for tablet 51221afb74e240ea9565f253d0c97e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:30 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:48854
I20260809 08:47:30.618765 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet 51221afb74e240ea9565f253d0c97e14 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:30 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 08:47:30.619089 14452 tablet_replica.cc:330] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: stopping tablet replica
I20260809 08:47:30.619204 14452 raft_consensus.cc:2229] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:30.619273 14452 raft_consensus.cc:2258] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:30.619822 13780 tablet_service.cc:1467] Processing CreateTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 (DEFAULT_TABLE table=test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.619949 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb060c0ec5cb480e80ee6e92f9ae0495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.620944 14379 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Bootstrap starting.
I20260809 08:47:30.620975 14238 tablet_service.cc:1467] Processing CreateTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 (DEFAULT_TABLE table=test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.621069 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb060c0ec5cb480e80ee6e92f9ae0495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.621546 14379 tablet_bootstrap.cc:654] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.622432 14378 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Bootstrap starting.
I20260809 08:47:30.619778 14005 tablet_service.cc:1467] Processing CreateTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 (DEFAULT_TABLE table=test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:30.622707 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb060c0ec5cb480e80ee6e92f9ae0495. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:30.622869 14378 tablet_bootstrap.cc:654] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.623445 14378 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: No bootstrap required, opened a new log
I20260809 08:47:30.623498 14378 ts_tablet_manager.cc:1397] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:30.623660 14378 raft_consensus.cc:348] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.623728 14378 raft_consensus.cc:374] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.623752 14378 raft_consensus.cc:729] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.623787 14378 consensus_queue.cc:260] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.623889 14378 ts_tablet_manager.cc:1428] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:30.624075 14384 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Bootstrap starting.
I20260809 08:47:30.624478 14384 tablet_bootstrap.cc:654] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:30.624617 14450 tablet_replica.cc:330] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: stopping tablet replica
I20260809 08:47:30.624737 14450 raft_consensus.cc:2229] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:30.624847 14450 raft_consensus.cc:2258] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:30.619652 14452 ts_tablet_manager.cc:1905] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:30.625138 14384 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: No bootstrap required, opened a new log
I20260809 08:47:30.625185 14384 ts_tablet_manager.cc:1397] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:30.625319 14384 raft_consensus.cc:348] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.625371 14384 raft_consensus.cc:374] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.625388 14384 raft_consensus.cc:729] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.625430 14384 consensus_queue.cc:260] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.625510 14384 ts_tablet_manager.cc:1428] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:30.626861 14451 tablet_replica.cc:330] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: stopping tablet replica
I20260809 08:47:30.626962 14451 raft_consensus.cc:2229] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:30.627020 14451 raft_consensus.cc:2258] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:30.627177 14452 ts_tablet_manager.cc:1918] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:30.627496 14451 ts_tablet_manager.cc:1905] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:30.628576 14451 ts_tablet_manager.cc:1918] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:30.628640 14451 log.cc:1196] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/51221afb74e240ea9565f253d0c97e14
I20260809 08:47:30.628866 14451 ts_tablet_manager.cc:1939] T 51221afb74e240ea9565f253d0c97e14 P 5af2b8ea5b444283851ab21fa2535204: Deleting consensus metadata
I20260809 08:47:30.629249 13559 catalog_manager.cc:4841] TS 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793): tablet 51221afb74e240ea9565f253d0c97e14 (table test-table [id=750574d2ea1a46edb1ba07986e673a60]) successfully deleted
I20260809 08:47:30.629374 14452 log.cc:1196] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/51221afb74e240ea9565f253d0c97e14
I20260809 08:47:30.629530 14450 ts_tablet_manager.cc:1905] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:30.629601 14452 ts_tablet_manager.cc:1939] T 51221afb74e240ea9565f253d0c97e14 P 160461e930b24fc9b87ece2a3cec94f7: Deleting consensus metadata
I20260809 08:47:30.629961 13564 catalog_manager.cc:4841] TS 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533): tablet 51221afb74e240ea9565f253d0c97e14 (table test-table [id=750574d2ea1a46edb1ba07986e673a60]) successfully deleted
I20260809 08:47:30.630209 14379 tablet_bootstrap.cc:492] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: No bootstrap required, opened a new log
I20260809 08:47:30.630255 14379 ts_tablet_manager.cc:1397] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260809 08:47:30.630364 14379 raft_consensus.cc:348] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.630407 14379 raft_consensus.cc:374] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:30.630419 14379 raft_consensus.cc:729] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Initialized, Role: FOLLOWER
I20260809 08:47:30.630451 14379 consensus_queue.cc:260] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.630537 14379 ts_tablet_manager.cc:1428] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:30.631237 14450 ts_tablet_manager.cc:1918] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:30.631309 14450 log.cc:1196] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/51221afb74e240ea9565f253d0c97e14
I20260809 08:47:30.631570 14450 ts_tablet_manager.cc:1939] T 51221afb74e240ea9565f253d0c97e14 P 551dc7ef45a34b6983afb55d490837cc: Deleting consensus metadata
I20260809 08:47:30.631909 13559 catalog_manager.cc:4841] TS 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433): tablet 51221afb74e240ea9565f253d0c97e14 (table test-table [id=750574d2ea1a46edb1ba07986e673a60]) successfully deleted
I20260809 08:47:30.968469 14385 raft_consensus.cc:482] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:30.968580 14385 raft_consensus.cc:504] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.968736 14385 leader_election.cc:288] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:30.969020 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb060c0ec5cb480e80ee6e92f9ae0495" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc" is_pre_election: true
I20260809 08:47:30.969020 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb060c0ec5cb480e80ee6e92f9ae0495" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7" is_pre_election: true
I20260809 08:47:30.969129 14037 raft_consensus.cc:2454] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 0.
I20260809 08:47:30.969130 14277 raft_consensus.cc:2454] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 0.
I20260809 08:47:30.969304 13714 leader_election.cc:302] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 551dc7ef45a34b6983afb55d490837cc, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:30.969409 14385 raft_consensus.cc:2781] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:30.969444 14385 raft_consensus.cc:482] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:30.969467 14385 raft_consensus.cc:3037] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.970136 14385 raft_consensus.cc:504] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.970273 14385 leader_election.cc:288] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 election: Requested vote from peers 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:30.970422 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb060c0ec5cb480e80ee6e92f9ae0495" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc"
I20260809 08:47:30.970439 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb060c0ec5cb480e80ee6e92f9ae0495" candidate_uuid: "5af2b8ea5b444283851ab21fa2535204" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7"
I20260809 08:47:30.970512 14037 raft_consensus.cc:3037] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.970512 14277 raft_consensus.cc:3037] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:30.971257 14277 raft_consensus.cc:2454] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 1.
I20260809 08:47:30.971257 14037 raft_consensus.cc:2454] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5af2b8ea5b444283851ab21fa2535204 in term 1.
I20260809 08:47:30.971417 13714 leader_election.cc:302] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551dc7ef45a34b6983afb55d490837cc, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:30.971524 14385 raft_consensus.cc:2781] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:30.971580 14385 raft_consensus.cc:686] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 1 LEADER]: Becoming Leader. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Running, Role: LEADER
I20260809 08:47:30.971619 14385 consensus_queue.cc:237] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:30.972149 13579 catalog_manager.cc:5495] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "5af2b8ea5b444283851ab21fa2535204" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:31.063225 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:31.061393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.10.184.65:34793)
0809 08:47:31.061433 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:31.061435 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:31.061444 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:31.061491 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:31.061492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:31.061500 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:31.061543 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:31.061630 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.062085 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.062754 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.062926 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.062968 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:31.063057 (+ 89us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:31.063080 (+ 23us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:31.063110 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:31.063148 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:47:31.063164 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:31.066823 14375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:31.065163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:39143 (local address 127.10.184.65:34793)
0809 08:47:31.065211 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:31.065213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:31.065219 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:47:31.065235 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:31.065236 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:31.065242 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:31.065281 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:31.065348 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.065765 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.066365 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.066524 (+ 159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.066561 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:31.066655 (+ 94us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:31.066675 (+ 20us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:31.066705 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:31.066736 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 08:47:31.066753 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":32}
I20260809 08:47:31.066871 14445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:31.065097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.10.184.65:39143)
0809 08:47:31.065131 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:31.065142 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:31.065184 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:31.065283 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:31.065284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:31.065285 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:31.065315 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:31.065316 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.065809 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.065809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:31.066313 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:31.066315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:31.066566 (+ 251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:31.066567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:31.066599 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:31.066604 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:31.066706 (+ 102us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:31.066707 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:31.066734 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 08:47:31.066769 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16,"mutex_wait_us":1}
I20260809 08:47:31.068451 14037 raft_consensus.cc:1261] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Refusing update from remote peer 5af2b8ea5b444283851ab21fa2535204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:31.068459 14277 raft_consensus.cc:1261] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Refusing update from remote peer 5af2b8ea5b444283851ab21fa2535204: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:31.068650 14437 consensus_queue.cc:1035] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:31.068751 14413 consensus_queue.cc:1035] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [LEADER]: Connected to new peer: Peer: permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:47:31.072796 14422 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:47:32.590183 13579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:32.590286 13579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52034:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:32.591055 13579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Sending DeleteTablet for 3 replicas of tablet cb060c0ec5cb480e80ee6e92f9ae0495
I20260809 08:47:32.591296 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:32 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 08:47:32.591326 13780 tablet_service.cc:1514] Processing DeleteTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:32 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:48854
I20260809 08:47:32.591480 14238 tablet_service.cc:1514] Processing DeleteTablet for tablet cb060c0ec5cb480e80ee6e92f9ae0495 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:32 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41660
I20260809 08:47:32.591655 14492 tablet_replica.cc:330] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: stopping tablet replica
I20260809 08:47:32.591701 14492 raft_consensus.cc:2229] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:32.591732 14492 raft_consensus.cc:2258] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:32.591976 14492 ts_tablet_manager.cc:1905] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:32.593200 14491 tablet_replica.cc:330] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: stopping tablet replica
I20260809 08:47:32.593272 14491 raft_consensus.cc:2229] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:32.593315 14491 raft_consensus.cc:2258] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:32.593513 14492 ts_tablet_manager.cc:1918] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:47:32.593539 14491 ts_tablet_manager.cc:1905] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:32.593578 14492 log.cc:1196] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/cb060c0ec5cb480e80ee6e92f9ae0495
I20260809 08:47:32.593789 14492 ts_tablet_manager.cc:1939] T cb060c0ec5cb480e80ee6e92f9ae0495 P 551dc7ef45a34b6983afb55d490837cc: Deleting consensus metadata
I20260809 08:47:32.594058 13559 catalog_manager.cc:4841] TS 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433): tablet cb060c0ec5cb480e80ee6e92f9ae0495 (table test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]) successfully deleted
I20260809 08:47:32.594812 14491 ts_tablet_manager.cc:1918] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:47:32.594866 14491 log.cc:1196] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/cb060c0ec5cb480e80ee6e92f9ae0495
I20260809 08:47:32.595048 14491 ts_tablet_manager.cc:1939] T cb060c0ec5cb480e80ee6e92f9ae0495 P 160461e930b24fc9b87ece2a3cec94f7: Deleting consensus metadata
I20260809 08:47:32.595196 14490 tablet_replica.cc:330] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: stopping tablet replica
I20260809 08:47:32.595249 14490 raft_consensus.cc:2229] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:32.595337 14490 raft_consensus.cc:2258] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:32.595422 13564 catalog_manager.cc:4841] TS 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533): tablet cb060c0ec5cb480e80ee6e92f9ae0495 (table test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]) successfully deleted
I20260809 08:47:32.595568 14490 ts_tablet_manager.cc:1905] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:32.596621 14490 ts_tablet_manager.cc:1918] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:47:32.596668 14490 log.cc:1196] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/cb060c0ec5cb480e80ee6e92f9ae0495
I20260809 08:47:32.596868 14490 ts_tablet_manager.cc:1939] T cb060c0ec5cb480e80ee6e92f9ae0495 P 5af2b8ea5b444283851ab21fa2535204: Deleting consensus metadata
I20260809 08:47:32.597170 13559 catalog_manager.cc:4841] TS 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793): tablet cb060c0ec5cb480e80ee6e92f9ae0495 (table test-table [id=2ecca5e7b0d0405cbc7ea0c3c3e58a93]) successfully deleted
Aug 09 08:47:32 dist-test-slave-ncp9 krb5kdc[13519](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:47:32 dist-test-slave-ncp9 krb5kdc[13519](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265252, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:32.613957 14502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:32.607464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.10.184.126:41885)
0809 08:47:32.607590 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:32.607591 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:32.607600 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:32.607654 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:32.607654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:32.607664 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:32.607721 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:32.608202 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:32.608718 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:32.609422 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:32.609624 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:32.609680 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:32.611623 (+ 1943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:32.611624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:32.611625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:32.611631 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:32.612865 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:32.613236 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:32.613237 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:32.613238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:32.613295 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:32.613443 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:32.613443 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:32.613444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:32.613564 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:32.613638 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:32.613792 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:47:32.613876 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":78,"threads_started":1}
I20260809 08:47:32.615229 13579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:32.615365 13579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:32.616572 13780 tablet_service.cc:1467] Processing CreateTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 (DEFAULT_TABLE table=test-table [id=b563c1a7052549eb93e051e81ae7ff3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:32.616603 14005 tablet_service.cc:1467] Processing CreateTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 (DEFAULT_TABLE table=test-table [id=b563c1a7052549eb93e051e81ae7ff3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:32.616592 14238 tablet_service.cc:1467] Processing CreateTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 (DEFAULT_TABLE table=test-table [id=b563c1a7052549eb93e051e81ae7ff3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:32.616707 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db3e7c673a84f0886f7b9c55391ebe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:32.616707 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db3e7c673a84f0886f7b9c55391ebe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:32.616724 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0db3e7c673a84f0886f7b9c55391ebe7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:32.617858 14505 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Bootstrap starting.
I20260809 08:47:32.618059 14511 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Bootstrap starting.
I20260809 08:47:32.618402 14511 tablet_bootstrap.cc:654] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:32.618520 14505 tablet_bootstrap.cc:654] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:32.618726 14506 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Bootstrap starting.
I20260809 08:47:32.619040 14511 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: No bootstrap required, opened a new log
I20260809 08:47:32.619050 14505 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: No bootstrap required, opened a new log
I20260809 08:47:32.619074 14506 tablet_bootstrap.cc:654] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:32.619099 14505 ts_tablet_manager.cc:1397] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:32.619104 14511 ts_tablet_manager.cc:1397] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:32.619216 14505 raft_consensus.cc:348] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619284 14505 raft_consensus.cc:374] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:32.619303 14505 raft_consensus.cc:729] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Initialized, Role: FOLLOWER
I20260809 08:47:32.619263 14511 raft_consensus.cc:348] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619338 14511 raft_consensus.cc:374] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:32.619360 14511 raft_consensus.cc:729] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Initialized, Role: FOLLOWER
I20260809 08:47:32.619367 14505 consensus_queue.cc:260] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619457 14511 consensus_queue.cc:260] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619534 14505 ts_tablet_manager.cc:1428] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:32.619627 14506 tablet_bootstrap.cc:492] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: No bootstrap required, opened a new log
I20260809 08:47:32.619638 14511 ts_tablet_manager.cc:1428] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:32.619675 14506 ts_tablet_manager.cc:1397] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:32.619767 14506 raft_consensus.cc:348] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619824 14506 raft_consensus.cc:374] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:32.619843 14506 raft_consensus.cc:729] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Initialized, Role: FOLLOWER
I20260809 08:47:32.619882 14506 consensus_queue.cc:260] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:32.619961 14506 ts_tablet_manager.cc:1428] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:33.005398 14519 raft_consensus.cc:482] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:33.005506 14519 raft_consensus.cc:504] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.005692 14519 leader_election.cc:288] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793), 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533)
I20260809 08:47:33.005947 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db3e7c673a84f0886f7b9c55391ebe7" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204" is_pre_election: true
I20260809 08:47:33.005947 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db3e7c673a84f0886f7b9c55391ebe7" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7" is_pre_election: true
I20260809 08:47:33.006068 13817 raft_consensus.cc:2454] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 0.
I20260809 08:47:33.006068 14037 raft_consensus.cc:2454] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 0.
I20260809 08:47:33.006250 14158 leader_election.cc:302] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 551dc7ef45a34b6983afb55d490837cc, 5af2b8ea5b444283851ab21fa2535204; no voters:
I20260809 08:47:33.006369 14519 raft_consensus.cc:2781] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:33.006426 14519 raft_consensus.cc:482] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:33.006440 14519 raft_consensus.cc:3037] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.007229 14519 raft_consensus.cc:504] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.007347 14519 leader_election.cc:288] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 election: Requested vote from peers 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793), 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533)
I20260809 08:47:33.007498 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db3e7c673a84f0886f7b9c55391ebe7" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204"
I20260809 08:47:33.007498 14037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0db3e7c673a84f0886f7b9c55391ebe7" candidate_uuid: "551dc7ef45a34b6983afb55d490837cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "160461e930b24fc9b87ece2a3cec94f7"
I20260809 08:47:33.007575 13817 raft_consensus.cc:3037] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.007575 14037 raft_consensus.cc:3037] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.008330 13817 raft_consensus.cc:2454] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 1.
I20260809 08:47:33.008332 14037 raft_consensus.cc:2454] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 551dc7ef45a34b6983afb55d490837cc in term 1.
I20260809 08:47:33.008510 14163 leader_election.cc:302] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 551dc7ef45a34b6983afb55d490837cc; no voters:
I20260809 08:47:33.008601 14519 raft_consensus.cc:2781] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:33.008761 14519 raft_consensus.cc:686] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 1 LEADER]: Becoming Leader. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Running, Role: LEADER
I20260809 08:47:33.008817 14519 consensus_queue.cc:237] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.010506 13578 catalog_manager.cc:5495] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 551dc7ef45a34b6983afb55d490837cc (127.10.184.67). New cstate: current_term: 1 leader_uuid: "551dc7ef45a34b6983afb55d490837cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } health_report { overall_health: HEALTHY } } }
I20260809 08:47:33.063727 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.058608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41688 (local address 127.10.184.67:33433)
0809 08:47:33.060719 (+ 2111us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.060721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.060731 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:33.060757 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.060758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.060771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.060845 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:33.061059 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.061832 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.062930 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.063248 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.063311 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.063463 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.063498 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.063543 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.063597 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.063618 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2046,"thread_start_us":113,"threads_started":1}
I20260809 08:47:33.067183 13817 raft_consensus.cc:1261] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Refusing update from remote peer 551dc7ef45a34b6983afb55d490837cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:33.067162 14037 raft_consensus.cc:1261] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Refusing update from remote peer 551dc7ef45a34b6983afb55d490837cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:33.067453 14522 consensus_queue.cc:1035] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:33.067627 14519 consensus_queue.cc:1035] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:33.079763 13579 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:33.079870 13579 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:33.080466 13579 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Sending DeleteTablet for 3 replicas of tablet 0db3e7c673a84f0886f7b9c55391ebe7
I20260809 08:47:33.080708 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:33 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 08:47:33.080746 13780 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:33 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:48854
I20260809 08:47:33.080821 14491 tablet_replica.cc:330] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: stopping tablet replica
I20260809 08:47:33.080809 14238 tablet_service.cc:1514] Processing DeleteTablet for tablet 0db3e7c673a84f0886f7b9c55391ebe7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:33 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41660
I20260809 08:47:33.080842 14490 tablet_replica.cc:330] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: stopping tablet replica
I20260809 08:47:33.080868 14491 raft_consensus.cc:2229] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:33.080886 14490 raft_consensus.cc:2229] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:33.080926 14491 raft_consensus.cc:2258] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:33.080924 14492 tablet_replica.cc:330] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: stopping tablet replica
I20260809 08:47:33.080945 14490 raft_consensus.cc:2258] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:33.080978 14492 raft_consensus.cc:2229] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:33.081030 14492 raft_consensus.cc:2258] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:33.081238 14491 ts_tablet_manager.cc:1905] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:33.081223 13579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 08:47:33.081349 14490 ts_tablet_manager.cc:1905] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:47:33.081362 13579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:33.082455 14005 tablet_service.cc:1467] Processing CreateTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 (DEFAULT_TABLE table=test-table [id=a539a99f0175408b9f7c8b127f9608ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:33.082566 14005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04bf3ddc09f48f59650ccd095c94d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:33.082944 14491 ts_tablet_manager.cc:1918] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:33.082980 14490 ts_tablet_manager.cc:1918] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:33.083000 14491 log.cc:1196] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/0db3e7c673a84f0886f7b9c55391ebe7
I20260809 08:47:33.083024 14490 log.cc:1196] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/0db3e7c673a84f0886f7b9c55391ebe7
I20260809 08:47:33.083204 14490 ts_tablet_manager.cc:1939] T 0db3e7c673a84f0886f7b9c55391ebe7 P 5af2b8ea5b444283851ab21fa2535204: Deleting consensus metadata
I20260809 08:47:33.083211 14491 ts_tablet_manager.cc:1939] T 0db3e7c673a84f0886f7b9c55391ebe7 P 160461e930b24fc9b87ece2a3cec94f7: Deleting consensus metadata
I20260809 08:47:33.083495 13564 catalog_manager.cc:4841] TS 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533): tablet 0db3e7c673a84f0886f7b9c55391ebe7 (table test-table [id=b563c1a7052549eb93e051e81ae7ff3e]) successfully deleted
I20260809 08:47:33.083720 14511 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Bootstrap starting.
I20260809 08:47:33.083976 14238 tablet_service.cc:1467] Processing CreateTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 (DEFAULT_TABLE table=test-table [id=a539a99f0175408b9f7c8b127f9608ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:33.084013 13559 catalog_manager.cc:4841] TS 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793): tablet 0db3e7c673a84f0886f7b9c55391ebe7 (table test-table [id=b563c1a7052549eb93e051e81ae7ff3e]) successfully deleted
I20260809 08:47:33.084007 13780 tablet_service.cc:1467] Processing CreateTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 (DEFAULT_TABLE table=test-table [id=a539a99f0175408b9f7c8b127f9608ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:33.084069 14238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04bf3ddc09f48f59650ccd095c94d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:33.084123 13780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a04bf3ddc09f48f59650ccd095c94d19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:33.084228 14511 tablet_bootstrap.cc:654] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:33.084368 14492 ts_tablet_manager.cc:1905] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:33.085299 14506 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204: Bootstrap starting.
I20260809 08:47:33.085716 14506 tablet_bootstrap.cc:654] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:33.085871 14492 ts_tablet_manager.cc:1918] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:33.085920 14492 log.cc:1196] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/0db3e7c673a84f0886f7b9c55391ebe7
I20260809 08:47:33.086140 14511 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: No bootstrap required, opened a new log
I20260809 08:47:33.086131 14492 ts_tablet_manager.cc:1939] T 0db3e7c673a84f0886f7b9c55391ebe7 P 551dc7ef45a34b6983afb55d490837cc: Deleting consensus metadata
I20260809 08:47:33.086195 14511 ts_tablet_manager.cc:1397] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:33.086283 14505 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Bootstrap starting.
I20260809 08:47:33.086375 14511 raft_consensus.cc:348] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.086445 14511 raft_consensus.cc:374] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:33.086463 14511 raft_consensus.cc:729] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Initialized, Role: FOLLOWER
I20260809 08:47:33.086508 14511 consensus_queue.cc:260] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.086592 14511 ts_tablet_manager.cc:1428] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:33.086609 14505 tablet_bootstrap.cc:654] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:33.086905 14506 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204: No bootstrap required, opened a new log
I20260809 08:47:33.086956 14506 ts_tablet_manager.cc:1397] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:33.087057 14506 raft_consensus.cc:348] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.087119 14506 raft_consensus.cc:374] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:33.087138 14506 raft_consensus.cc:729] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5af2b8ea5b444283851ab21fa2535204, State: Initialized, Role: FOLLOWER
I20260809 08:47:33.087175 14506 consensus_queue.cc:260] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.087260 14506 ts_tablet_manager.cc:1428] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:33.089505 13559 catalog_manager.cc:4841] TS 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433): tablet 0db3e7c673a84f0886f7b9c55391ebe7 (table test-table [id=b563c1a7052549eb93e051e81ae7ff3e]) successfully deleted
I20260809 08:47:33.089684 14505 tablet_bootstrap.cc:492] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: No bootstrap required, opened a new log
I20260809 08:47:33.089749 14505 ts_tablet_manager.cc:1397] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 08:47:33.089867 14505 raft_consensus.cc:348] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.089949 14505 raft_consensus.cc:374] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:33.089980 14505 raft_consensus.cc:729] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 551dc7ef45a34b6983afb55d490837cc, State: Initialized, Role: FOLLOWER
I20260809 08:47:33.090042 14505 consensus_queue.cc:260] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.090142 14505 ts_tablet_manager.cc:1428] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:33.213055 14521 raft_consensus.cc:482] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:33.213184 14521 raft_consensus.cc:504] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.213593 14521 leader_election.cc:288] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:33.216765 14526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.214040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:39739 (local address 127.10.184.67:33433)
0809 08:47:33.214083 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.214085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.214140 (+ 55us) server_negotiation.cc:407] Connection header received
0809 08:47:33.214219 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.214220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.214232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.214278 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:33.214480 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.215013 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216076 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.216442 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216504 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.216515 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.216658 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.216700 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:33.216771 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.213836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:48861 (local address 127.10.184.65:34793)
0809 08:47:33.214174 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.214177 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.214185 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:33.214214 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.214215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.214226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.214293 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:33.214507 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.215013 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216071 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.216456 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216520 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.216528 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.216661 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.216700 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":52,"threads_started":1}
I20260809 08:47:33.216785 14540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.213780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.10.184.66:48861)
0809 08:47:33.213900 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:33.213911 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:33.213965 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:33.214346 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:33.214347 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:33.214350 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:33.214428 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.214430 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.215112 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.215113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.215982 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.215984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.216524 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.216597 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.216600 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:33.216645 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:47:33.216693 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"mutex_wait_us":38,"thread_start_us":67,"threads_started":1}
I20260809 08:47:33.216779 14543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.213956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33433 (local address 127.10.184.66:39739)
0809 08:47:33.214080 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:33.214090 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:33.214153 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:33.214286 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:33.214287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:33.214290 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:33.214414 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.214417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.215090 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.215090 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.215986 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.215988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.216517 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.216518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.216592 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.216597 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:33.216643 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:47:33.216685 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":50,"thread_start_us":83,"threads_started":1}
I20260809 08:47:33.217322 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04bf3ddc09f48f59650ccd095c94d19" candidate_uuid: "160461e930b24fc9b87ece2a3cec94f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204" is_pre_election: true
I20260809 08:47:33.217322 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04bf3ddc09f48f59650ccd095c94d19" candidate_uuid: "160461e930b24fc9b87ece2a3cec94f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc" is_pre_election: true
I20260809 08:47:33.217415 14277 raft_consensus.cc:2454] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 160461e930b24fc9b87ece2a3cec94f7 in term 0.
I20260809 08:47:33.217415 13817 raft_consensus.cc:2454] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 160461e930b24fc9b87ece2a3cec94f7 in term 0.
I20260809 08:47:33.217593 13930 leader_election.cc:302] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 551dc7ef45a34b6983afb55d490837cc; no voters:
I20260809 08:47:33.217721 14521 raft_consensus.cc:2781] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:33.217761 14521 raft_consensus.cc:482] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:33.217784 14521 raft_consensus.cc:3037] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.218444 14521 raft_consensus.cc:504] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.218546 14521 leader_election.cc:288] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [CANDIDATE]: Term 1 election: Requested vote from peers 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793), 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433)
I20260809 08:47:33.218698 13817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04bf3ddc09f48f59650ccd095c94d19" candidate_uuid: "160461e930b24fc9b87ece2a3cec94f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5af2b8ea5b444283851ab21fa2535204"
I20260809 08:47:33.218739 14277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a04bf3ddc09f48f59650ccd095c94d19" candidate_uuid: "160461e930b24fc9b87ece2a3cec94f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "551dc7ef45a34b6983afb55d490837cc"
I20260809 08:47:33.218783 13817 raft_consensus.cc:3037] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.218799 14277 raft_consensus.cc:3037] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:33.219338 14277 raft_consensus.cc:2454] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 160461e930b24fc9b87ece2a3cec94f7 in term 1.
I20260809 08:47:33.219435 13817 raft_consensus.cc:2454] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 160461e930b24fc9b87ece2a3cec94f7 in term 1.
I20260809 08:47:33.219499 13930 leader_election.cc:302] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 160461e930b24fc9b87ece2a3cec94f7, 551dc7ef45a34b6983afb55d490837cc; no voters:
I20260809 08:47:33.219581 14521 raft_consensus.cc:2781] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:33.219731 14521 raft_consensus.cc:686] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 LEADER]: Becoming Leader. State: Replica: 160461e930b24fc9b87ece2a3cec94f7, State: Running, Role: LEADER
I20260809 08:47:33.219796 14521 consensus_queue.cc:237] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } }
I20260809 08:47:33.220460 13580 catalog_manager.cc:5495] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66). New cstate: current_term: 1 leader_uuid: "160461e930b24fc9b87ece2a3cec94f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "160461e930b24fc9b87ece2a3cec94f7" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41533 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:33.238385 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.236198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51776 (local address 127.10.184.66:41533)
0809 08:47:33.236338 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.236340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.236346 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:47:33.236377 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.236377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.236394 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.236463 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:33.236561 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.237079 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.237717 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.237911 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.237953 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.238041 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.238203 (+ 162us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.238260 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.238298 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.238314 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260809 08:47:33.242674 14542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.240584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:38879 (local address 127.10.184.65:34793)
0809 08:47:33.240640 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.240642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.240663 (+ 21us) server_negotiation.cc:407] Connection header received
0809 08:47:33.240686 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.240687 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.240696 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.240741 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:33.240840 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.241333 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.242026 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.242236 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.242288 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.242451 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.242485 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.242524 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.242576 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.242596 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:47:33.242697 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.240446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.10.184.66:38879)
0809 08:47:33.240587 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:33.240597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:33.240646 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:33.240741 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:33.240742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:33.240743 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:33.240796 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.240798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.241402 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.241403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.241956 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.241958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.242315 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.242316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.242373 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.242385 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.242535 (+ 150us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.242536 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:33.242575 (+ 39us) client_negotiation.cc:240] Negotiation successful
0809 08:47:33.242611 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":58,"threads_started":1}
I20260809 08:47:33.244387 14277 raft_consensus.cc:1261] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Refusing update from remote peer 160461e930b24fc9b87ece2a3cec94f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:33.244405 13817 raft_consensus.cc:1261] T a04bf3ddc09f48f59650ccd095c94d19 P 5af2b8ea5b444283851ab21fa2535204 [term 1 FOLLOWER]: Refusing update from remote peer 160461e930b24fc9b87ece2a3cec94f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:33.244604 14521 consensus_queue.cc:1035] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "551dc7ef45a34b6983afb55d490837cc" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:33.244947 14545 consensus_queue.cc:1035] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5af2b8ea5b444283851ab21fa2535204" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:33.251298 14550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.249120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:47405 (local address 127.10.184.66:41533)
0809 08:47:33.249182 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:33.249184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:33.249367 (+ 183us) server_negotiation.cc:407] Connection header received
0809 08:47:33.249391 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:33.249391 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:33.249401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:33.249449 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:33.249563 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.249977 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.250660 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.250904 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.250959 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.251129 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.251155 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.251197 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:33.251241 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 08:47:33.251258 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:47:33.251322 14560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:33.249083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41533 (local address 127.10.184.65:47405)
0809 08:47:33.249282 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:33.249289 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:33.249327 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:33.249467 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:33.249467 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:33.249469 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:33.249510 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.249511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.250022 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.250022 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.250575 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:33.250577 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:33.250972 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:33.250973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:33.251031 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:33.251035 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:33.251208 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:33.251210 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:33.251238 (+ 28us) client_negotiation.cc:240] Negotiation successful
0809 08:47:33.251269 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":90,"threads_started":1}
W20260809 08:47:33.251608 14472 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 08:47:34.769686 13580 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:34.769788 13580 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:34.770627 13580 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 36411063254048a99175d93ab7304cd3: Sending DeleteTablet for 3 replicas of tablet a04bf3ddc09f48f59650ccd095c94d19
I20260809 08:47:34.770915 13780 tablet_service.cc:1514] Processing DeleteTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:34 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:48854
I20260809 08:47:34.770915 14238 tablet_service.cc:1514] Processing DeleteTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:34 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41660
I20260809 08:47:34.771080 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13694
I20260809 08:47:34.771099 14005 tablet_service.cc:1514] Processing DeleteTablet for tablet a04bf3ddc09f48f59650ccd095c94d19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:34 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:51766
I20260809 08:47:34.779584 14598 tablet_replica.cc:330] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: stopping tablet replica
I20260809 08:47:34.779670 14598 raft_consensus.cc:2229] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:34.779780 14598 raft_consensus.cc:2258] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:34.780045 14598 ts_tablet_manager.cc:1905] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:34.780972 14596 tablet_replica.cc:330] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: stopping tablet replica
I20260809 08:47:34.781037 14596 raft_consensus.cc:2229] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:34.781082 14596 raft_consensus.cc:2258] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:34.781311 14596 ts_tablet_manager.cc:1905] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:34.781648 14598 ts_tablet_manager.cc:1918] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:47:34.781714 14598 log.cc:1196] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/a04bf3ddc09f48f59650ccd095c94d19
I20260809 08:47:34.781944 14598 ts_tablet_manager.cc:1939] T a04bf3ddc09f48f59650ccd095c94d19 P 160461e930b24fc9b87ece2a3cec94f7: Deleting consensus metadata
I20260809 08:47:34.782387 13564 catalog_manager.cc:4841] TS 160461e930b24fc9b87ece2a3cec94f7 (127.10.184.66:41533): tablet a04bf3ddc09f48f59650ccd095c94d19 (table test-table [id=a539a99f0175408b9f7c8b127f9608ea]) successfully deleted
I20260809 08:47:34.782873 14596 ts_tablet_manager.cc:1918] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 08:47:34.782938 14596 log.cc:1196] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/a04bf3ddc09f48f59650ccd095c94d19
I20260809 08:47:34.783136 14596 ts_tablet_manager.cc:1939] T a04bf3ddc09f48f59650ccd095c94d19 P 551dc7ef45a34b6983afb55d490837cc: Deleting consensus metadata
I20260809 08:47:34.783455 13559 catalog_manager.cc:4841] TS 551dc7ef45a34b6983afb55d490837cc (127.10.184.67:33433): tablet a04bf3ddc09f48f59650ccd095c94d19 (table test-table [id=a539a99f0175408b9f7c8b127f9608ea]) successfully deleted
W20260809 08:47:34.786890 13559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:34793 (error 108)
I20260809 08:47:34.787122 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13901
I20260809 08:47:34.789336 14603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:34.787288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34793 (local address 127.0.0.1:48878)
0809 08:47:34.789233 (+ 1945us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:34.789294 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:34793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1916,"thread_start_us":62,"threads_started":1}
W20260809 08:47:34.789449 13559 catalog_manager.cc:4584] TS 5af2b8ea5b444283851ab21fa2535204 (127.10.184.65:34793): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a04bf3ddc09f48f59650ccd095c94d19: Network error: Client connection negotiation failed: client connection to 127.10.184.65:34793: connect: Connection refused (error 111)
I20260809 08:47:34.794986 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 14131
I20260809 08:47:34.805284 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13534
2026-08-09T08:47:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6627 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:34 dist-test-slave-ncp9 krb5kdc[14606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:34 dist-test-slave-ncp9 krb5kdc[14606](info): set up 2 sockets
Aug 09 08:47:34 dist-test-slave-ncp9 krb5kdc[14606](info): commencing operation
krb5kdc: starting...
W20260809 08:47:35.930595 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:36.043427 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36495
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:36495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:36.121033 14620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:36.121223 14620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:36.121253 14620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:36.122370 14620 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:36.122413 14620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:36.122433 14620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:36.122448 14620 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:36.122463 14620 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:36.123904 14620 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:36495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:36.124131 14620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:36.124336 14620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:36.128866 14629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.128937 14630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.128872 14634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:36.129120 14620 server_base.cc:1034] running on GCE node
I20260809 08:47:36.129328 14620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:36.129603 14620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:36.130743 14620 hybrid_clock.cc:648] HybridClock initialized: now 1786265256130728 us; error 27 us; skew 500 ppm
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:36.133010 14620 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:36.133397 14620 webserver.cc:460] Webserver started at http://127.10.184.126:44599/ using document root <none> and password file <none>
I20260809 08:47:36.133617 14620 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:36.133672 14620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:36.133798 14620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:36.134758 14620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "cef84484b80543f884cc7c18f9394e92"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "404912c6da9d6b40944821aa7931cd5f"
server_key_iv: "cc4b989d39bf7a8dffe3bef6316112d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.135076 14620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "cef84484b80543f884cc7c18f9394e92"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "404912c6da9d6b40944821aa7931cd5f"
server_key_iv: "cc4b989d39bf7a8dffe3bef6316112d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.136375 14620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:36.137043 14644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.137194 14620 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:36.137279 14620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "cef84484b80543f884cc7c18f9394e92"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "404912c6da9d6b40944821aa7931cd5f"
server_key_iv: "cc4b989d39bf7a8dffe3bef6316112d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.137346 14620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:36.155925 14620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:36.157606 14620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:36.157742 14620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:36.161391 14620 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:36495
I20260809 08:47:36.161433 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:36495 every 8 connection(s)
I20260809 08:47:36.161748 14620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:36.162582 14715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:36.164752 14715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92: Bootstrap starting.
I20260809 08:47:36.165313 14715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:36.165578 14715 log.cc:824] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:36.166220 14715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92: No bootstrap required, opened a new log
I20260809 08:47:36.167284 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 14620
I20260809 08:47:36.167371 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:36.167515 10977 external_mini_cluster.cc:1428] Setting key 6a6338ecf0b7416abe620b80531be775
I20260809 08:47:36.167848 14715 raft_consensus.cc:348] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } }
I20260809 08:47:36.168030 14715 raft_consensus.cc:374] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:36.168100 14715 raft_consensus.cc:729] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cef84484b80543f884cc7c18f9394e92, State: Initialized, Role: FOLLOWER
I20260809 08:47:36.168278 14715 consensus_queue.cc:260] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } }
I20260809 08:47:36.168355 14715 raft_consensus.cc:388] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:36.168377 14715 raft_consensus.cc:482] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:36.168426 14715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:36.169407 14715 raft_consensus.cc:504] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } }
I20260809 08:47:36.169497 14715 leader_election.cc:302] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cef84484b80543f884cc7c18f9394e92; no voters:
I20260809 08:47:36.169641 14715 leader_election.cc:288] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:36.169704 14723 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:36.169871 14723 raft_consensus.cc:686] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [term 1 LEADER]: Becoming Leader. State: Replica: cef84484b80543f884cc7c18f9394e92, State: Running, Role: LEADER
I20260809 08:47:36.169880 14715 sys_catalog.cc:564] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:36.170001 14723 consensus_queue.cc:237] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } }
I20260809 08:47:36.170439 14725 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cef84484b80543f884cc7c18f9394e92. Latest consensus state: current_term: 1 leader_uuid: "cef84484b80543f884cc7c18f9394e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } } }
I20260809 08:47:36.170545 14725 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:36.170738 14733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:36.170449 14724 sys_catalog.cc:455] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cef84484b80543f884cc7c18f9394e92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cef84484b80543f884cc7c18f9394e92" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36495 } } }
I20260809 08:47:36.170849 14724 sys_catalog.cc:458] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:36.171310 14733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:36.172927 14733 catalog_manager.cc:1347] Generated new cluster ID: 200563ce482d478a80e71798953b3ef0
I20260809 08:47:36.172977 14733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:36.178997 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.168027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.10.184.126:36495)
0809 08:47:36.168203 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:36.168213 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:36.168243 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:47:36.168352 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:36.168359 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:36.168413 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:36.168549 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:36.169201 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.171056 (+ 1855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.172218 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.172769 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.173192 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.175511 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:36.175515 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:36.175516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:36.175528 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:36.177464 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.177827 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.177830 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.177831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.177952 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.178125 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.178126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.178126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.178426 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:36.178582 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:36.178663 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 08:47:36.178802 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":86,"threads_started":1}
I20260809 08:47:36.183210 14733 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:36.183892 14733 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:36.192705 14733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cef84484b80543f884cc7c18f9394e92: Generated new TSK 0
I20260809 08:47:36.192863 14733 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:36.219457 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:36.298818 14753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:36.299002 14753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:36.299023 14753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:36.300096 14753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:36.300127 14753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:36.300151 14753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:36.301523 14753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:36.301712 14753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:36.301911 14753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:36.306545 14760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.306558 14766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.306614 14770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:36.306779 14753 server_base.cc:1034] running on GCE node
I20260809 08:47:36.306988 14753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:36.307207 14753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:36.308341 14753 hybrid_clock.cc:648] HybridClock initialized: now 1786265256308328 us; error 32 us; skew 500 ppm
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:36.310672 14753 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:36.311046 14753 webserver.cc:460] Webserver started at http://127.10.184.65:36173/ using document root <none> and password file <none>
I20260809 08:47:36.311232 14753 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:36.311280 14753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:36.311399 14753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:36.312340 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "4b47516fcc544e919204057d5e11e71b"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "da15ed02a0fdd60aaa94e6b6b7162771"
server_key_iv: "254761fb5d2f8a215f43abcc62f97634"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.312664 14753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "4b47516fcc544e919204057d5e11e71b"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "da15ed02a0fdd60aaa94e6b6b7162771"
server_key_iv: "254761fb5d2f8a215f43abcc62f97634"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.313894 14753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:47:36.314543 14781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.314689 14753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.314770 14753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "4b47516fcc544e919204057d5e11e71b"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "da15ed02a0fdd60aaa94e6b6b7162771"
server_key_iv: "254761fb5d2f8a215f43abcc62f97634"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.314833 14753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:36.321980 14753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:36.323541 14753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:36.323675 14753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:36.323858 14753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:36.324108 14753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:36.324139 14753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.324164 14753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:36.324182 14753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.329484 14753 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:36031
I20260809 08:47:36.329519 14945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:36031 every 8 connection(s)
I20260809 08:47:36.329780 14753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:36.333791 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 14753
I20260809 08:47:36.333886 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:36.334030 10977 external_mini_cluster.cc:1428] Setting key f03fc7288ad7fc2080becc9c9d3c0d5b
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:36.338567 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.330374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:51639 (local address 127.10.184.126:36495)
0809 08:47:36.330414 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:36.330416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:36.330595 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:47:36.331033 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:36.331034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:36.331050 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:36.331099 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:36.331964 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.332463 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.333473 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.333707 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.333778 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.336041 (+ 2263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:36.336043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:36.336043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:36.336049 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:36.337350 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.337717 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.337718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.337719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.337784 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.337981 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.337981 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.337982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.338105 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:36.338181 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:36.338386 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:47:36.338479 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:36.338603 14951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.330347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36495 (local address 127.10.184.65:51639)
0809 08:47:36.330492 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:36.330524 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:36.330940 (+ 416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:36.331163 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:36.331167 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:36.331699 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:36.331869 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.331874 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.332545 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.332546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.333391 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.333394 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.333795 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.333796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.333920 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.334392 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:36.334402 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:36.335959 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:36.337430 (+ 1471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.337444 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.337445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.337647 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.337842 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.337843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.337843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.337931 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.338178 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:36.338181 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:36.338279 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 08:47:36.338351 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:36.338461 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":70,"threads_started":1}
I20260809 08:47:36.339046 14947 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36495
I20260809 08:47:36.339133 14947 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:36.339291 14947 heartbeater.cc:511] Master 127.10.184.126:36495 requested a full tablet report, sending...
I20260809 08:47:36.339836 14668 ts_manager.cc:194] Registered new tserver with Master: 4b47516fcc544e919204057d5e11e71b (127.10.184.65:36031)
I20260809 08:47:36.340410 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:51639
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:36.356987 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:36.435426 14956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:36.435604 14956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:36.435626 14956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:36.436735 14956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:36.436774 14956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:36.436801 14956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:36.438225 14956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:36.438432 14956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:36.438632 14956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:36.443461 14967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:36.443521 14956 server_base.cc:1034] running on GCE node
W20260809 08:47:36.443514 14969 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.443461 14963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:36.443882 14956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:36.444098 14956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:36.445233 14956 hybrid_clock.cc:648] HybridClock initialized: now 1786265256445212 us; error 39 us; skew 500 ppm
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:36.447489 14956 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:36.447894 14956 webserver.cc:460] Webserver started at http://127.10.184.66:34887/ using document root <none> and password file <none>
I20260809 08:47:36.448094 14956 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:36.448140 14956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:36.448256 14956 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:36.449221 14956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "d47e732f51c64d2eb261bb1691f2be33"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "94b27b7db22c2b6c0aa8e106942da606"
server_key_iv: "34654d996b269e9c12c7ab742d7d2b8c"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.449528 14956 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "d47e732f51c64d2eb261bb1691f2be33"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "94b27b7db22c2b6c0aa8e106942da606"
server_key_iv: "34654d996b269e9c12c7ab742d7d2b8c"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.450763 14956 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:36.451356 14981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.451504 14956 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:36.451584 14956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "d47e732f51c64d2eb261bb1691f2be33"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "94b27b7db22c2b6c0aa8e106942da606"
server_key_iv: "34654d996b269e9c12c7ab742d7d2b8c"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.451640 14956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:36.467654 14956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:36.469280 14956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:36.469404 14956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:36.469581 14956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:36.469851 14956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:36.469882 14956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.469908 14956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:36.469925 14956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.475229 14956 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:39949
I20260809 08:47:36.475270 15151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:39949 every 8 connection(s)
I20260809 08:47:36.475518 14956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:36.480932 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 14956
I20260809 08:47:36.481035 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:36.481230 10977 external_mini_cluster.cc:1428] Setting key be985157980601462082cb2cbe078c2c
I20260809 08:47:36.483911 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.476135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:58767 (local address 127.10.184.126:36495)
0809 08:47:36.476190 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:36.476192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:36.476379 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:47:36.476794 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:36.476794 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:36.476803 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:36.476853 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:36.477683 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.478169 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.479043 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.479223 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.479264 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.481251 (+ 1987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:36.481253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:36.481253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:36.481260 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:36.482602 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.483018 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.483018 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.483019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.483065 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.483298 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.483299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.483299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.483391 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:36.483449 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:36.483738 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 08:47:36.483825 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:47:36.483965 15155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.476107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36495 (local address 127.10.184.66:58767)
0809 08:47:36.476268 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:36.476307 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:36.476722 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:36.476916 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:36.476919 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:36.477446 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:36.477618 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.477623 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.478229 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.478230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.478986 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.478988 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.479273 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.479274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.479369 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.479741 (+ 372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:36.479748 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:36.481131 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:36.482675 (+ 1544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.482682 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.482684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.482953 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.483116 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.483117 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.483117 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.483248 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.483448 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:36.483453 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:36.483610 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 08:47:36.483704 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 08:47:36.483819 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":82,"threads_started":1}
I20260809 08:47:36.484378 15152 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36495
I20260809 08:47:36.484475 15152 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:36.484668 15152 heartbeater.cc:511] Master 127.10.184.126:36495 requested a full tablet report, sending...
I20260809 08:47:36.485036 14668 ts_manager.cc:194] Registered new tserver with Master: d47e732f51c64d2eb261bb1691f2be33 (127.10.184.66:39949)
I20260809 08:47:36.485379 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:58767
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:36.505168 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:36.588312 15160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:36.588512 15160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:36.588543 15160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:36.589686 15160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:36.589728 15160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:36.589762 15160 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:36.591137 15160 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:36.591367 15160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:36.591573 15160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:36.596279 15166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.596292 15170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:36.596293 15168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:36.596382 15160 server_base.cc:1034] running on GCE node
I20260809 08:47:36.596690 15160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:36.596961 15160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:36.598104 15160 hybrid_clock.cc:648] HybridClock initialized: now 1786265256598092 us; error 25 us; skew 500 ppm
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:36.600354 15160 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:36.600776 15160 webserver.cc:460] Webserver started at http://127.10.184.67:41873/ using document root <none> and password file <none>
I20260809 08:47:36.601011 15160 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:36.601061 15160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:36.601197 15160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:36.602200 15160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "dd044f5c96494a1a9ad29e4b6415461f"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "693e5ce8e1de4d2e1d9a974b3e85dbcd"
server_key_iv: "b6b752a54199c9512817735fd23febf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.602537 15160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "dd044f5c96494a1a9ad29e4b6415461f"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "693e5ce8e1de4d2e1d9a974b3e85dbcd"
server_key_iv: "b6b752a54199c9512817735fd23febf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.603745 15160 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:36.604399 15180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.604553 15160 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:36.604647 15160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "dd044f5c96494a1a9ad29e4b6415461f"
format_stamp: "Formatted at 2026-08-09 08:47:36 on dist-test-slave-ncp9"
server_key: "693e5ce8e1de4d2e1d9a974b3e85dbcd"
server_key_iv: "b6b752a54199c9512817735fd23febf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:36.604712 15160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:36.620481 15160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:36.622291 15160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:36.622426 15160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:36.622628 15160 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:36.622906 15160 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:36.622941 15160 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.622968 15160 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:36.622984 15160 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:36.628473 15160 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:35439
I20260809 08:47:36.628528 15352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:35439 every 8 connection(s)
I20260809 08:47:36.628808 15160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:36.633065 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 15160
I20260809 08:47:36.633183 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:36.633378 10977 external_mini_cluster.cc:1428] Setting key 431476c2cbf4670437b0bd6114aff1e7
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[14606](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265256, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:36.637305 14722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.629517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:43907 (local address 127.10.184.126:36495)
0809 08:47:36.629564 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:36.629566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:36.629758 (+ 192us) server_negotiation.cc:407] Connection header received
0809 08:47:36.630195 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:36.630196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:36.630206 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:36.630254 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:36.631101 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.631585 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.632497 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.632703 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.632753 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.634951 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:36.634952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:36.634953 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:36.634959 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:36.636197 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.636496 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.636497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.636497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.636543 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:36.636712 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:36.636713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:36.636713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:36.636805 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:36.636849 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:36.637138 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 08:47:36.637217 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:36.637454 15359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:36.629483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36495 (local address 127.10.184.67:43907)
0809 08:47:36.629645 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:36.629684 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:36.630120 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:36.630307 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:36.630310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:36.630854 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:36.631033 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.631038 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.631643 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.631643 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.632426 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:36.632428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:36.632761 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:36.632762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:36.632855 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:36.633383 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:36.633392 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:36.634872 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:36.636256 (+ 1384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.636258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.636260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.636446 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.636586 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:36.636586 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:36.636586 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:36.636672 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:36.636846 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:36.636849 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:36.636996 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 08:47:36.637101 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 08:47:36.637259 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":83,"threads_started":1}
I20260809 08:47:36.637828 15355 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36495
I20260809 08:47:36.637917 15355 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:36.638079 15355 heartbeater.cc:511] Master 127.10.184.126:36495 requested a full tablet report, sending...
I20260809 08:47:36.638393 14668 ts_manager.cc:194] Registered new tserver with Master: dd044f5c96494a1a9ad29e4b6415461f (127.10.184.67:35439)
I20260809 08:47:36.638726 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:43907
I20260809 08:47:36.644881 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:47:36.656574 14722 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:36.655785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.10.184.126:36495)
0809 08:47:36.655841 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:36.655844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:36.655932 (+ 88us) server_negotiation.cc:407] Connection header received
0809 08:47:36.655982 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:36.655982 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:36.655991 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:36.656040 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:36.656510 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54272: BlockingRecv error: recv got EOF from 127.0.0.1:54272 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:47:36.657601 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 14753
I20260809 08:47:36.692878 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 14956
I20260809 08:47:36.701067 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 15160
I20260809 08:47:36.710963 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 14620
2026-08-09T08:47:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1904 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[15374](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[15374](info): set up 2 sockets
Aug 09 08:47:36 dist-test-slave-ncp9 krb5kdc[15374](info): commencing operation
krb5kdc: starting...
W20260809 08:47:37.828591 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:37 dist-test-slave-ncp9 krb5kdc[15374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:37.939532 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39019
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39019
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:38.019323 15395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:38.019532 15395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:38.019553 15395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:38.020699 15395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:38.020740 15395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:38.020753 15395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:38.020777 15395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:38.020799 15395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:38.022437 15395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39019
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:38.022708 15395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:38.022913 15395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:47:38.027714 15395 server_base.cc:1034] running on GCE node
W20260809 08:47:38.027645 15404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:38.027642 15401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:38.027659 15402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:38.028111 15395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:38.028384 15395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:38.029546 15395 hybrid_clock.cc:648] HybridClock initialized: now 1786265258029531 us; error 29 us; skew 500 ppm
Aug 09 08:47:38 dist-test-slave-ncp9 krb5kdc[15374](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265258, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:38.034854 15395 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:38.035295 15395 webserver.cc:460] Webserver started at http://127.10.184.126:38191/ using document root <none> and password file <none>
I20260809 08:47:38.035506 15395 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:38.035554 15395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:38.035682 15395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:38.036686 15395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "142488540853445dbcb68e05bddd6d31"
format_stamp: "Formatted at 2026-08-09 08:47:38 on dist-test-slave-ncp9"
server_key: "99b4e63b3a8fa35a45f3f64fbfc44340"
server_key_iv: "0e8db7a4da27fb17a72465616cb1337f"
server_key_version: "encryptionkey@0"
I20260809 08:47:38.037056 15395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "142488540853445dbcb68e05bddd6d31"
format_stamp: "Formatted at 2026-08-09 08:47:38 on dist-test-slave-ncp9"
server_key: "99b4e63b3a8fa35a45f3f64fbfc44340"
server_key_iv: "0e8db7a4da27fb17a72465616cb1337f"
server_key_version: "encryptionkey@0"
I20260809 08:47:38.038472 15395 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:38.039108 15416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:38.039261 15395 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:38.039348 15395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "142488540853445dbcb68e05bddd6d31"
format_stamp: "Formatted at 2026-08-09 08:47:38 on dist-test-slave-ncp9"
server_key: "99b4e63b3a8fa35a45f3f64fbfc44340"
server_key_iv: "0e8db7a4da27fb17a72465616cb1337f"
server_key_version: "encryptionkey@0"
I20260809 08:47:38.039409 15395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:38.052589 15395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:38.054682 15395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:38.054818 15395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:38.058617 15395 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39019
I20260809 08:47:38.058668 15501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39019 every 8 connection(s)
I20260809 08:47:38.059090 15395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:38.059725 15504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:38.062026 15504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31: Bootstrap starting.
I20260809 08:47:38.062592 15504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:38.062831 15504 log.cc:824] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:38.063433 15504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31: No bootstrap required, opened a new log
I20260809 08:47:38.063689 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 15395
I20260809 08:47:38.063788 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:38.063936 10977 external_mini_cluster.cc:1428] Setting key b39ecc1110a589706fd9dc6595ee696a
I20260809 08:47:38.066015 15504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } }
I20260809 08:47:38.066155 15504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:38.066191 15504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 142488540853445dbcb68e05bddd6d31, State: Initialized, Role: FOLLOWER
I20260809 08:47:38.066336 15504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } }
I20260809 08:47:38.066421 15504 raft_consensus.cc:388] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:38.066457 15504 raft_consensus.cc:482] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:38.066510 15504 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:38.067049 15504 raft_consensus.cc:504] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } }
I20260809 08:47:38.067154 15504 leader_election.cc:302] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 142488540853445dbcb68e05bddd6d31; no voters:
I20260809 08:47:38.067313 15504 leader_election.cc:288] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:38.067368 15509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:38.067538 15509 raft_consensus.cc:686] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [term 1 LEADER]: Becoming Leader. State: Replica: 142488540853445dbcb68e05bddd6d31, State: Running, Role: LEADER
I20260809 08:47:38.067581 15504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:38.067628 15509 consensus_queue.cc:237] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } }
I20260809 08:47:38.067987 15509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 142488540853445dbcb68e05bddd6d31. Latest consensus state: current_term: 1 leader_uuid: "142488540853445dbcb68e05bddd6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } } }
I20260809 08:47:38.068077 15509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:38.068236 15510 sys_catalog.cc:455] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "142488540853445dbcb68e05bddd6d31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "142488540853445dbcb68e05bddd6d31" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39019 } } }
I20260809 08:47:38.068318 15510 sys_catalog.cc:458] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:38.069036 15522 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:38.069561 15522 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:38.071208 15522 catalog_manager.cc:1347] Generated new cluster ID: 5c5b5b4908664760b88bfaea6b95b032
I20260809 08:47:38.071269 15522 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:38 dist-test-slave-ncp9 krb5kdc[15374](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:38.075012 15508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:38.064356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.10.184.126:39019)
0809 08:47:38.064564 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:38.064573 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:38.064609 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:47:38.065997 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:38.066005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:38.066057 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:38.066182 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:38.066820 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:38.067971 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:38.069033 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:38.069625 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:38.070044 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:38.072130 (+ 2086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:38.072135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:38.072135 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:38.072147 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:38.073651 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:38.073969 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:38.073973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:38.073973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:38.074069 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:38.074244 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:38.074244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:38.074245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:38.074460 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:38.074606 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:38.074699 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:47:38.074826 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260809 08:47:38.075935 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:38.079572 15522 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:38.080204 15522 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:38.089388 15508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:38.086302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.10.184.126:39019)
0809 08:47:38.086339 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:38.086341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:38.086461 (+ 120us) server_negotiation.cc:407] Connection header received
0809 08:47:38.086517 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:38.086518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:38.086529 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:38.086580 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:38.086917 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:38.087398 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:38.088085 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:38.088304 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:38.088356 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:38.088992 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:38.088994 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:38.088994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:38.089156 (+ 162us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:38.089180 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:38.089185 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:38.089232 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:38.089295 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:47:38.089319 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:38.093766 15522 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 142488540853445dbcb68e05bddd6d31: Generated new TSK 0
I20260809 08:47:38.093930 15522 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:38.097587 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 15395
2026-08-09T08:47:38Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1390 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:38 dist-test-slave-ncp9 krb5kdc[15554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:38 dist-test-slave-ncp9 krb5kdc[15554](info): set up 2 sockets
Aug 09 08:47:38 dist-test-slave-ncp9 krb5kdc[15554](info): commencing operation
krb5kdc: starting...
W20260809 08:47:39.238937 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:39.347733 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36445
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:36445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:39.426299 15572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:39.426498 15572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:39.426527 15572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:39.427636 15572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:39.427676 15572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:39.427695 15572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:39.427711 15572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:39.427726 15572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:39.429191 15572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:36445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:39.429423 15572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:39.429625 15572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:47:39.434407 15572 server_base.cc:1034] running on GCE node
W20260809 08:47:39.434355 15582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.434398 15580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.434367 15578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:39.434870 15572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:39.435135 15572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:39.436272 15572 hybrid_clock.cc:648] HybridClock initialized: now 1786265259436259 us; error 28 us; skew 500 ppm
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:39.438598 15572 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:39.439085 15572 webserver.cc:460] Webserver started at http://127.10.184.126:41135/ using document root <none> and password file <none>
I20260809 08:47:39.439275 15572 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:39.439322 15572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:39.439446 15572 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:39.440395 15572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "d47d81e1613846058a8b4a5bc782e1bd"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "920e2378b7dd3b499b649e019e1dd013"
server_key_iv: "d848a5f7d3a93dbd9ce35ffd3c8d7da5"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.440724 15572 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "d47d81e1613846058a8b4a5bc782e1bd"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "920e2378b7dd3b499b649e019e1dd013"
server_key_iv: "d848a5f7d3a93dbd9ce35ffd3c8d7da5"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.441936 15572 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:39.442593 15593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.442829 15572 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:39.442986 15572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "d47d81e1613846058a8b4a5bc782e1bd"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "920e2378b7dd3b499b649e019e1dd013"
server_key_iv: "d848a5f7d3a93dbd9ce35ffd3c8d7da5"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.443068 15572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:39.461715 15572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:39.463666 15572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:39.463773 15572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:39.467463 15572 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:36445
I20260809 08:47:39.467525 15671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:36445 every 8 connection(s)
I20260809 08:47:39.467849 15572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:39.468995 15672 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:39.471099 15672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd: Bootstrap starting.
I20260809 08:47:39.471626 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 15572
I20260809 08:47:39.471688 15672 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:39.471741 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:39.471922 10977 external_mini_cluster.cc:1428] Setting key b82409529df71163b14eb42bb437fa39
I20260809 08:47:39.471979 15672 log.cc:824] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:39.472661 15672 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd: No bootstrap required, opened a new log
I20260809 08:47:39.474618 15672 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } }
I20260809 08:47:39.474757 15672 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:39.474790 15672 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47d81e1613846058a8b4a5bc782e1bd, State: Initialized, Role: FOLLOWER
I20260809 08:47:39.474900 15672 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } }
I20260809 08:47:39.474970 15672 raft_consensus.cc:388] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:39.475001 15672 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:39.475049 15672 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:39.475546 15672 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } }
I20260809 08:47:39.475643 15672 leader_election.cc:302] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d47d81e1613846058a8b4a5bc782e1bd; no voters:
I20260809 08:47:39.475798 15672 leader_election.cc:288] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:39.475832 15677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:39.475978 15677 raft_consensus.cc:686] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [term 1 LEADER]: Becoming Leader. State: Replica: d47d81e1613846058a8b4a5bc782e1bd, State: Running, Role: LEADER
I20260809 08:47:39.476029 15672 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:39.476100 15677 consensus_queue.cc:237] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } }
I20260809 08:47:39.476531 15679 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d47d81e1613846058a8b4a5bc782e1bd. Latest consensus state: current_term: 1 leader_uuid: "d47d81e1613846058a8b4a5bc782e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } } }
I20260809 08:47:39.476528 15678 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d47d81e1613846058a8b4a5bc782e1bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47d81e1613846058a8b4a5bc782e1bd" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36445 } } }
I20260809 08:47:39.476686 15679 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:39.477438 15678 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:39.477507 15699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:39.478224 15699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:39.479822 15699 catalog_manager.cc:1347] Generated new cluster ID: f976cfe6796645fcb69decf043b0e58a
I20260809 08:47:39.479875 15699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:39.482018 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:39.472357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34602 (local address 127.10.184.126:36445)
0809 08:47:39.472522 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:39.472532 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:39.472562 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:47:39.473049 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:39.473057 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:39.473103 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:39.473223 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:39.473816 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.474644 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.475523 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.476100 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.476240 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:39.479011 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:39.479015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:39.479016 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:39.479028 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:39.480586 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.480927 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.480930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.480931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.481017 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.481201 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.481202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.481203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.481422 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:39.481563 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:39.481688 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 08:47:39.481816 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 08:47:39.538327 15699 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:39.538839 15699 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:39.541321 15699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d47d81e1613846058a8b4a5bc782e1bd: Generated new TSK 0
I20260809 08:47:39.541471 15699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:39.578960 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:39.657712 15711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:39.657895 15711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:39.657917 15711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:39.659003 15711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:39.659040 15711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:39.659068 15711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:39.660449 15711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:39.660660 15711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:39.660863 15711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:39.665469 15719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.665550 15720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.665616 15722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:39.665676 15711 server_base.cc:1034] running on GCE node
I20260809 08:47:39.665907 15711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:39.666113 15711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:39.667253 15711 hybrid_clock.cc:648] HybridClock initialized: now 1786265259667244 us; error 31 us; skew 500 ppm
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:39.669620 15711 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:39.669993 15711 webserver.cc:460] Webserver started at http://127.10.184.65:34833/ using document root <none> and password file <none>
I20260809 08:47:39.670184 15711 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:39.670231 15711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:39.670349 15711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:39.671314 15711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "8d492d9d11f740f3a3f2d31c84e5c96a"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "ba7ebc062f9d5964090b0ab4ffeab2e8"
server_key_iv: "4bd9c2585e4a91a6f71dc76abf9376f8"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.671633 15711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "8d492d9d11f740f3a3f2d31c84e5c96a"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "ba7ebc062f9d5964090b0ab4ffeab2e8"
server_key_iv: "4bd9c2585e4a91a6f71dc76abf9376f8"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.672825 15711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:39.673537 15730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.673672 15711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:39.673753 15711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "8d492d9d11f740f3a3f2d31c84e5c96a"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "ba7ebc062f9d5964090b0ab4ffeab2e8"
server_key_iv: "4bd9c2585e4a91a6f71dc76abf9376f8"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.673812 15711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:39.685892 15711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:39.687422 15711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:39.687542 15711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:39.687729 15711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:39.687979 15711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:39.688011 15711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.688036 15711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:39.688060 15711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.693594 15711 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:44361
I20260809 08:47:39.693631 15901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:44361 every 8 connection(s)
I20260809 08:47:39.693894 15711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:39.701925 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:39.694529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:33127 (local address 127.10.184.126:36445)
0809 08:47:39.694584 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:39.694585 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:39.694767 (+ 182us) server_negotiation.cc:407] Connection header received
0809 08:47:39.695200 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:39.695201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:39.695209 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:39.695261 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:39.696089 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.696595 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.697552 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.697744 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.697793 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:39.699642 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:39.699644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:39.699644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:39.699651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:39.700883 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.701243 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.701244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.701244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.701296 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.701463 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.701464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.701464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.701557 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:39.701600 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:39.701796 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 08:47:39.701862 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:47:39.702052 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:39.694486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36445 (local address 127.10.184.65:33127)
0809 08:47:39.694645 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:39.694692 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:39.695123 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:39.695317 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:39.695320 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:39.695846 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:39.696025 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:39.696029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.696660 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.696660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:39.697489 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:39.697492 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.697802 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.697803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:39.697894 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:39.698282 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:39.698289 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:39.699548 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:39.700973 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:39.700979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:39.700981 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:39.701183 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:39.701337 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:39.701338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:39.701338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:39.701422 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:39.701596 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:39.701599 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:39.701708 (+ 109us) client_negotiation.cc:769] Sending connection context
0809 08:47:39.701781 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:47:39.701889 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":84,"threads_started":1}
I20260809 08:47:39.702533 15904 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36445
I20260809 08:47:39.702618 15904 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:39.702759 15904 heartbeater.cc:511] Master 127.10.184.126:36445 requested a full tablet report, sending...
I20260809 08:47:39.703217 15622 ts_manager.cc:194] Registered new tserver with Master: 8d492d9d11f740f3a3f2d31c84e5c96a (127.10.184.65:44361)
I20260809 08:47:39.703457 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 15711
I20260809 08:47:39.703542 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:39.703703 10977 external_mini_cluster.cc:1428] Setting key 9054962c05b7734e2321209ed5c098c2
I20260809 08:47:39.703963 15622 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:33127
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:39.727049 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:39.806712 15913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:39.806900 15913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:39.806921 15913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:39.808022 15913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:39.808060 15913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:39.808089 15913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:39.809537 15913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:39.809747 15913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:39.809942 15913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:39.814556 15920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.814594 15924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.814618 15919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:39.814905 15913 server_base.cc:1034] running on GCE node
I20260809 08:47:39.815146 15913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:39.815390 15913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:39.816546 15913 hybrid_clock.cc:648] HybridClock initialized: now 1786265259816520 us; error 35 us; skew 500 ppm
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:39.818982 15913 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:39.819419 15913 webserver.cc:460] Webserver started at http://127.10.184.66:38385/ using document root <none> and password file <none>
I20260809 08:47:39.819619 15913 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:39.819671 15913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:39.819795 15913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:39.820792 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "20e61ca58979469797771b6c18c9aeb5"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "d9f4b7cdc1305c2e445de3176adb329e"
server_key_iv: "c1cac61d307fc3883338f902900abafb"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.821210 15913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "20e61ca58979469797771b6c18c9aeb5"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "d9f4b7cdc1305c2e445de3176adb329e"
server_key_iv: "c1cac61d307fc3883338f902900abafb"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.822435 15913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:39.823081 15934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.823227 15913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:39.823313 15913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "20e61ca58979469797771b6c18c9aeb5"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "d9f4b7cdc1305c2e445de3176adb329e"
server_key_iv: "c1cac61d307fc3883338f902900abafb"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.823374 15913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:39.842522 15913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:39.844154 15913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:39.844291 15913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:39.844491 15913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:39.844785 15913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:39.844822 15913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.844852 15913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:39.844879 15913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.850842 15913 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:42561
I20260809 08:47:39.850875 16107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:42561 every 8 connection(s)
I20260809 08:47:39.851167 15913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:39.851327 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 15913
I20260809 08:47:39.851441 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:39.851642 10977 external_mini_cluster.cc:1428] Setting key f3de9de7eb1a76046e77c93d40f118b4
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
I20260809 08:47:39.860821 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:39.851928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:35075 (local address 127.10.184.126:36445)
0809 08:47:39.851982 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:39.851985 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:39.852242 (+ 257us) server_negotiation.cc:407] Connection header received
0809 08:47:39.852791 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:39.852792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:39.852802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:39.852852 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:39.854015 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.854548 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.855754 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.855944 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.856009 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:39.858221 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:39.858223 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:39.858223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:39.858231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:39.859515 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.859894 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.859896 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.859896 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.859970 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:39.860198 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:39.860199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:39.860200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:39.860312 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:39.860375 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:39.860638 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 08:47:39.860725 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:47:39.860980 16113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:39.851909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36445 (local address 127.10.184.66:35075)
0809 08:47:39.852101 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:39.852145 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:39.852694 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:39.852941 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:39.852945 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:39.853651 (+ 706us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:39.853912 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:39.853919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.854644 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.854645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:39.855682 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:39.855685 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:39.856031 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:39.856033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:39.856146 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:39.856598 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:39.856606 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:39.858123 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:39.859604 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:39.859611 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:39.859613 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:39.859824 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:39.860030 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:39.860031 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:39.860031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:39.860138 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:39.860371 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:39.860377 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:39.860515 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 08:47:39.860607 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:47:39.860756 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":95,"threads_started":1}
I20260809 08:47:39.861405 16108 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36445
I20260809 08:47:39.861523 16108 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:39.861691 16108 heartbeater.cc:511] Master 127.10.184.126:36445 requested a full tablet report, sending...
I20260809 08:47:39.862033 15621 ts_manager.cc:194] Registered new tserver with Master: 20e61ca58979469797771b6c18c9aeb5 (127.10.184.66:42561)
I20260809 08:47:39.862401 15621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:35075
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:39.875385 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:39.954150 16118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:39.954334 16118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:39.954355 16118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:39.955397 16118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:39.955432 16118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:39.955456 16118 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:39.956849 16118 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:39.957057 16118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:39.957235 16118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:39.961860 16126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:39.961896 16118 server_base.cc:1034] running on GCE node
W20260809 08:47:39.961869 16130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:39.961876 16124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:39.962251 16118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:39.962473 16118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:39.963608 16118 hybrid_clock.cc:648] HybridClock initialized: now 1786265259963594 us; error 28 us; skew 500 ppm
Aug 09 08:47:39 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:39.966037 16118 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:39.966470 16118 webserver.cc:460] Webserver started at http://127.10.184.67:40495/ using document root <none> and password file <none>
I20260809 08:47:39.966660 16118 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:39.966710 16118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:39.966830 16118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:39.967787 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "372d7816c2f9445582599ccfd8254f51"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "e22f5f8c0b8ecec16b77139ee3e483d7"
server_key_iv: "36e0c80882f86b7fd6bcbe4a43c98266"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.968109 16118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "372d7816c2f9445582599ccfd8254f51"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "e22f5f8c0b8ecec16b77139ee3e483d7"
server_key_iv: "36e0c80882f86b7fd6bcbe4a43c98266"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.969563 16118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:39.970252 16140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:39.970413 16118 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:39.970495 16118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "372d7816c2f9445582599ccfd8254f51"
format_stamp: "Formatted at 2026-08-09 08:47:39 on dist-test-slave-ncp9"
server_key: "e22f5f8c0b8ecec16b77139ee3e483d7"
server_key_iv: "36e0c80882f86b7fd6bcbe4a43c98266"
server_key_version: "encryptionkey@0"
I20260809 08:47:39.970559 16118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:39.997902 16118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:39.999939 16118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:40.000066 16118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:40.000265 16118 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:40.000541 16118 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:40.000576 16118 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:40.000600 16118 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:40.000618 16118 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:40.006000 16118 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:41933
I20260809 08:47:40.006033 16312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:41933 every 8 connection(s)
I20260809 08:47:40.006318 16118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:40.009815 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 16118
I20260809 08:47:40.009898 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:40.010040 10977 external_mini_cluster.cc:1428] Setting key c80575a621a4e4eb415d39b4c9cea9fd
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265259, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:40.014740 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:40.006947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:36167 (local address 127.10.184.126:36445)
0809 08:47:40.007000 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:40.007002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:40.007188 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:47:40.007666 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:40.007667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:40.007677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:40.007726 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:40.008676 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.009210 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.010251 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.010524 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.010587 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:40.012502 (+ 1915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:40.012503 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:40.012504 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:40.012510 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:40.013783 (+ 1273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:40.014084 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:40.014085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:40.014085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:40.014131 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:40.014296 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:40.014296 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:40.014296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:40.014388 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:40.014431 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:40.014618 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 08:47:40.014679 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:47:40.014917 16317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:40.006915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36445 (local address 127.10.184.67:36167)
0809 08:47:40.007069 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:40.007108 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:40.007558 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:40.007812 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:40.007816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:40.008370 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:40.008585 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:40.008590 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.009289 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.009290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:40.010170 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:40.010173 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.010616 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.010617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:40.010762 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:40.011154 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:40.011163 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:40.012415 (+ 1252us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:40.013838 (+ 1423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:40.013841 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:40.013843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:40.014032 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:40.014172 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:40.014173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:40.014173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:40.014255 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:40.014427 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:40.014429 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:40.014531 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 08:47:40.014602 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:47:40.014721 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260809 08:47:40.015340 16314 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36445
I20260809 08:47:40.015421 16314 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:40.015576 16314 heartbeater.cc:511] Master 127.10.184.126:36445 requested a full tablet report, sending...
I20260809 08:47:40.015908 15621 ts_manager.cc:194] Registered new tserver with Master: 372d7816c2f9445582599ccfd8254f51 (127.10.184.67:41933)
I20260809 08:47:40.016242 15621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:36167
I20260809 08:47:40.021498 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[15554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[15554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265260, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:40.042827 15676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:40.036717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34618 (local address 127.10.184.126:36445)
0809 08:47:40.036768 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:40.036769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:40.036856 (+ 87us) server_negotiation.cc:407] Connection header received
0809 08:47:40.036926 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:40.036927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:40.036937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:40.036997 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:40.037481 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.037948 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.038590 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:40.038755 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:40.038794 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:40.040719 (+ 1925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:40.040721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:40.040721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:40.040728 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:40.041968 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:40.042259 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:40.042259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:40.042260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:40.042306 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:40.042434 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:40.042434 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:40.042434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:40.042561 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:40.042632 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:40.042706 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:47:40.042763 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260809 08:47:40.043038 15621 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34618
I20260809 08:47:40.043431 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 15711
I20260809 08:47:40.057354 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 15913
I20260809 08:47:40.063241 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 16118
I20260809 08:47:40.073333 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 15572
2026-08-09T08:47:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1972 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[16331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[16331](info): set up 2 sockets
Aug 09 08:47:40 dist-test-slave-ncp9 krb5kdc[16331](info): commencing operation
krb5kdc: starting...
W20260809 08:47:41.203199 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:41.315397 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:42619
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:42619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:41.394104 16351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:41.394300 16351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:41.394332 16351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:41.395447 16351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:41.395485 16351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:41.395503 16351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:41.395520 16351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:41.395536 16351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:41.397045 16351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:42619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:42619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:41.397281 16351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:41.397471 16351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:41.402014 16357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.402024 16360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.402117 16356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:41.402302 16351 server_base.cc:1034] running on GCE node
I20260809 08:47:41.402552 16351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:41.402819 16351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:41.403990 16351 hybrid_clock.cc:648] HybridClock initialized: now 1786265261403975 us; error 28 us; skew 500 ppm
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:41.406490 16351 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:41.406922 16351 webserver.cc:460] Webserver started at http://127.10.184.126:38117/ using document root <none> and password file <none>
I20260809 08:47:41.407109 16351 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:41.407155 16351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:41.407277 16351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:41.408342 16351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "b54f9cad2c4f00bdc2c48ec61bf3953a"
server_key_iv: "f4a7c57d37df9fdb7857995cf7b74a4f"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.408720 16351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "b54f9cad2c4f00bdc2c48ec61bf3953a"
server_key_iv: "f4a7c57d37df9fdb7857995cf7b74a4f"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.410014 16351 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:41.410665 16369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.410807 16351 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:41.410892 16351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "b54f9cad2c4f00bdc2c48ec61bf3953a"
server_key_iv: "f4a7c57d37df9fdb7857995cf7b74a4f"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.410948 16351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:41.439603 16351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:41.441843 16351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:41.441979 16351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:41.446165 16351 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:42619
I20260809 08:47:41.446655 16351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:41.447284 16449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:42619 every 8 connection(s)
I20260809 08:47:41.448320 16450 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:41.450201 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 16351
I20260809 08:47:41.450313 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:41.450486 10977 external_mini_cluster.cc:1428] Setting key 9f65b68706652a97e8eea4ec31d9bf10
I20260809 08:47:41.451591 16450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3: Bootstrap starting.
I20260809 08:47:41.452237 16450 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:41.452543 16450 log.cc:824] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:41.453380 16450 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3: No bootstrap required, opened a new log
I20260809 08:47:41.455943 16450 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } }
I20260809 08:47:41.456118 16450 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:41.456163 16450 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef95b3514cb4db9aba0c0c05b6d48e3, State: Initialized, Role: FOLLOWER
I20260809 08:47:41.456290 16450 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } }
I20260809 08:47:41.456372 16450 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:41.456406 16450 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:41.456444 16450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:41.457098 16450 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } }
I20260809 08:47:41.457206 16450 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef95b3514cb4db9aba0c0c05b6d48e3; no voters:
I20260809 08:47:41.457366 16450 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:41.457412 16458 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:41.457595 16458 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [term 1 LEADER]: Becoming Leader. State: Replica: 1ef95b3514cb4db9aba0c0c05b6d48e3, State: Running, Role: LEADER
I20260809 08:47:41.457644 16450 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:41.457702 16458 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } }
I20260809 08:47:41.458036 16458 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef95b3514cb4db9aba0c0c05b6d48e3. Latest consensus state: current_term: 1 leader_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } } }
I20260809 08:47:41.458122 16458 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:41.458302 16462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95b3514cb4db9aba0c0c05b6d48e3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42619 } } }
I20260809 08:47:41.458366 16462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:41.458865 16471 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:41.459379 16471 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:41.461000 16471 catalog_manager.cc:1347] Generated new cluster ID: 1ce9d1c86b954564aa9e8a1e9b53bc85
I20260809 08:47:41.461061 16471 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:41.465561 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.453003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53650 (local address 127.10.184.126:42619)
0809 08:47:41.453161 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:41.453170 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:41.453204 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:47:41.453842 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:41.453853 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:41.453920 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:41.454086 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:41.454990 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.456583 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.458063 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.458872 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.459076 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.462191 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:41.462195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:41.462196 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:41.462207 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:41.464174 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.464488 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.464491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.464491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.464572 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.464749 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.464749 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.464750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.465012 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:41.465176 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:41.465243 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:47:41.465369 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260809 08:47:41.474677 16471 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:41.475294 16471 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:41.484212 16471 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef95b3514cb4db9aba0c0c05b6d48e3: Generated new TSK 0
I20260809 08:47:41.484371 16471 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:41.513037 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:41.628141 16495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:41.628338 16495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:41.628360 16495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:41.629515 16495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:41.629556 16495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:41.629585 16495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:41.631068 16495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:41.631283 16495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:41.631536 16495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:41.636399 16504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.636399 16502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.636422 16508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:41.636605 16495 server_base.cc:1034] running on GCE node
I20260809 08:47:41.636801 16495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:41.637036 16495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:41.638185 16495 hybrid_clock.cc:648] HybridClock initialized: now 1786265261638168 us; error 29 us; skew 500 ppm
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:41.640547 16495 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:41.641045 16495 webserver.cc:460] Webserver started at http://127.10.184.65:44995/ using document root <none> and password file <none>
I20260809 08:47:41.641268 16495 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:41.641314 16495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:41.641441 16495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:41.642469 16495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "adcd6452cb9841b5ac56c754f517529e"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "7624c38f5e84bd0878b2f2900a918c05"
server_key_iv: "c0120642131d3f6cb747acf7e10ffa6a"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.642874 16495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "adcd6452cb9841b5ac56c754f517529e"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "7624c38f5e84bd0878b2f2900a918c05"
server_key_iv: "c0120642131d3f6cb747acf7e10ffa6a"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.644196 16495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:41.644855 16517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.645008 16495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:41.645093 16495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "adcd6452cb9841b5ac56c754f517529e"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "7624c38f5e84bd0878b2f2900a918c05"
server_key_iv: "c0120642131d3f6cb747acf7e10ffa6a"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.645153 16495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:41.655881 16495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:41.657630 16495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:41.657761 16495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:41.657972 16495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:41.658268 16495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:41.658303 16495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.658331 16495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:41.658355 16495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.664011 16495 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:41719
I20260809 08:47:41.664053 16696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:41719 every 8 connection(s)
I20260809 08:47:41.664352 16495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:41.668576 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 16495
I20260809 08:47:41.668676 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:41.668840 10977 external_mini_cluster.cc:1428] Setting key 5c0ee9a574ae97225298d8ba20bba62f
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:41.673722 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.665097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:51177 (local address 127.10.184.126:42619)
0809 08:47:41.665143 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:41.665145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:41.665348 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:47:41.666280 (+ 932us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:41.666282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:41.666295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:41.666346 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:41.666955 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.667465 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.668468 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.668676 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.668735 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.670994 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:41.670996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:41.670996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:41.671003 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:41.672431 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.672801 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.672803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.672803 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.672869 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.673115 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.673116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.673116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.673244 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:41.673301 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:41.673535 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 08:47:41.673622 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:41.673771 16701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.665080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42619 (local address 127.10.184.65:51177)
0809 08:47:41.665232 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:41.665268 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:41.666167 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:41.666418 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:41.666422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:41.666673 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:41.666856 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.666861 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.667548 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.667549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.668362 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.668365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.668797 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.668798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.668939 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.669453 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:41.669463 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:41.670906 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:41.672516 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.672522 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.672524 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.672732 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.672962 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.672963 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.672963 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.673061 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.673299 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:41.673303 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:41.673408 (+ 105us) client_negotiation.cc:769] Sending connection context
0809 08:47:41.673486 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:41.673599 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":77,"threads_started":1}
I20260809 08:47:41.674289 16697 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42619
I20260809 08:47:41.674374 16697 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:41.674546 16697 heartbeater.cc:511] Master 127.10.184.126:42619 requested a full tablet report, sending...
I20260809 08:47:41.675046 16396 ts_manager.cc:194] Registered new tserver with Master: adcd6452cb9841b5ac56c754f517529e (127.10.184.65:41719)
I20260809 08:47:41.675628 16396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:51177
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:41.693820 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:41.775063 16704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:41.775267 16704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:41.775300 16704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:41.776429 16704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:41.776469 16704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:41.776504 16704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:41.777999 16704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:41.778236 16704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:41.778440 16704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:41.783253 16714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.783344 16712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.783226 16716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:41.783442 16704 server_base.cc:1034] running on GCE node
I20260809 08:47:41.783674 16704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:41.783874 16704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:41.785008 16704 hybrid_clock.cc:648] HybridClock initialized: now 1786265261784994 us; error 29 us; skew 500 ppm
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:41.787329 16704 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:41.787753 16704 webserver.cc:460] Webserver started at http://127.10.184.66:41535/ using document root <none> and password file <none>
I20260809 08:47:41.787974 16704 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:41.788022 16704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:41.788153 16704 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:41.789193 16704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "a0fdbe0c98a642f5a2c5b559e0393e30"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "15b6f2ddf488deabdd8140857d5d9a18"
server_key_iv: "91c90cff32a2c40185107472d7907bf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.789520 16704 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "a0fdbe0c98a642f5a2c5b559e0393e30"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "15b6f2ddf488deabdd8140857d5d9a18"
server_key_iv: "91c90cff32a2c40185107472d7907bf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.790828 16704 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:41.791462 16726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.791605 16704 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:41.791678 16704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "a0fdbe0c98a642f5a2c5b559e0393e30"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "15b6f2ddf488deabdd8140857d5d9a18"
server_key_iv: "91c90cff32a2c40185107472d7907bf5"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.791736 16704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:41.805689 16704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:41.807432 16704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:41.807559 16704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:41.807762 16704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:41.808048 16704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:41.808079 16704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.808107 16704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:41.808135 16704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.813678 16704 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43963
I20260809 08:47:41.813735 16898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43963 every 8 connection(s)
I20260809 08:47:41.814019 16704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:41.818631 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 16704
I20260809 08:47:41.818725 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:41.818885 10977 external_mini_cluster.cc:1428] Setting key 3f9cd8f7dea2f481f7ab6aaf5777b032
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:41.823251 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.814757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:59543 (local address 127.10.184.126:42619)
0809 08:47:41.814814 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:41.814816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:41.815004 (+ 188us) server_negotiation.cc:407] Connection header received
0809 08:47:41.815937 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:41.815938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:41.815951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:41.816002 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:41.816615 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.817190 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.818192 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.818393 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.818450 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.820522 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:41.820524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:41.820524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:41.820531 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:41.822002 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.822394 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.822395 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.822395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.822459 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.822659 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.822660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.822660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.822804 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:41.822870 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:41.823085 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 08:47:41.823164 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:47:41.823335 16902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.814713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42619 (local address 127.10.184.66:59543)
0809 08:47:41.814888 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:41.814929 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:41.815833 (+ 904us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:41.816072 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:41.816077 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:41.816335 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:41.816523 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.816527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.817282 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.817284 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.818101 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.818103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.818463 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.818464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.818592 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.819102 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:41.819112 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:41.820433 (+ 1321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:41.822080 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.822086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.822088 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.822319 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.822519 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.822520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.822520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.822610 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.822858 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:41.822861 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:41.822972 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:47:41.823046 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:47:41.823168 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":89,"threads_started":1}
I20260809 08:47:41.823817 16899 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42619
I20260809 08:47:41.823941 16899 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:41.824132 16899 heartbeater.cc:511] Master 127.10.184.126:42619 requested a full tablet report, sending...
I20260809 08:47:41.824532 16396 ts_manager.cc:194] Registered new tserver with Master: a0fdbe0c98a642f5a2c5b559e0393e30 (127.10.184.66:43963)
I20260809 08:47:41.825019 16396 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:59543
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:41.843322 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:41.922936 16905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:41.923143 16905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:41.923180 16905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:41.924311 16905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:41.924351 16905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:41.924386 16905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:41.925916 16905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:41.926149 16905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:41.926366 16905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:41.931216 16914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:41.931231 16916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:41.931231 16905 server_base.cc:1034] running on GCE node
W20260809 08:47:41.931216 16919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:41.931618 16905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:41.931861 16905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:41.932948 16905 hybrid_clock.cc:648] HybridClock initialized: now 1786265261932935 us; error 27 us; skew 500 ppm
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:41.935303 16905 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:41.935690 16905 webserver.cc:460] Webserver started at http://127.10.184.67:46827/ using document root <none> and password file <none>
I20260809 08:47:41.935878 16905 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:41.935925 16905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:41.936040 16905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:41.937054 16905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "12cf32e82dfc4958a35a65bd4c822e85"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "ad060ba8ef9e8a7e33b697d659dc39f8"
server_key_iv: "97e3b9ba55d89ed990b93da1c34fc2d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.937404 16905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "12cf32e82dfc4958a35a65bd4c822e85"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "ad060ba8ef9e8a7e33b697d659dc39f8"
server_key_iv: "97e3b9ba55d89ed990b93da1c34fc2d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.938759 16905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:41.939380 16929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.939522 16905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:41.939602 16905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "12cf32e82dfc4958a35a65bd4c822e85"
format_stamp: "Formatted at 2026-08-09 08:47:41 on dist-test-slave-ncp9"
server_key: "ad060ba8ef9e8a7e33b697d659dc39f8"
server_key_iv: "97e3b9ba55d89ed990b93da1c34fc2d9"
server_key_version: "encryptionkey@0"
I20260809 08:47:41.939661 16905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:41.950919 16905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:41.952468 16905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:41.952594 16905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:41.952781 16905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:41.953091 16905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:41.953140 16905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.953177 16905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:41.953202 16905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:41.958745 16905 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:45073
I20260809 08:47:41.958811 17098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:45073 every 8 connection(s)
I20260809 08:47:41.959075 16905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:41.967332 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.959789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48955 (local address 127.10.184.126:42619)
0809 08:47:41.959847 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:41.959850 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:41.960034 (+ 184us) server_negotiation.cc:407] Connection header received
0809 08:47:41.960920 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:41.960921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:41.960933 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:41.960981 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:41.961514 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.962005 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.962907 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.963080 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.963122 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.965019 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:41.965020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:41.965021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:41.965028 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:41.966110 (+ 1082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.966471 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.966472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.966473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.966526 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:41.966744 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:41.966745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:41.966745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:41.966856 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:41.966913 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:41.967161 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 08:47:41.967245 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:47:41.967487 17101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.959773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42619 (local address 127.10.184.67:48955)
0809 08:47:41.959931 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:41.959963 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:41.960800 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:41.961043 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:41.961046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:41.961280 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:41.961452 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.961456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.962063 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.962063 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.962850 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:41.962852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.963128 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.963129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:41.963219 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.963611 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:41.963619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:41.964926 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:41.966190 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.966196 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.966198 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.966402 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.966582 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:41.966583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:41.966583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:41.966686 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:41.966911 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:41.966917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:41.967046 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 08:47:41.967130 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:47:41.967286 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":85,"threads_started":1}
I20260809 08:47:41.967717 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 16905
I20260809 08:47:41.967820 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:41.967984 10977 external_mini_cluster.cc:1428] Setting key 872c2182c5b4a054199cbdfc73f613d2
I20260809 08:47:41.967964 17099 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42619
I20260809 08:47:41.968051 17099 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:41.968232 17099 heartbeater.cc:511] Master 127.10.184.126:42619 requested a full tablet report, sending...
I20260809 08:47:41.968603 16400 ts_manager.cc:194] Registered new tserver with Master: 12cf32e82dfc4958a35a65bd4c822e85 (127.10.184.67:45073)
I20260809 08:47:41.969049 16400 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:48955
I20260809 08:47:41.979481 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:47:41 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:42.001848 17111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:41.994896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.10.184.65:41719)
0809 08:47:41.995071 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:41.995076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:41.995087 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:41.995113 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:41.995116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:41.995129 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:41.995192 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:41.995738 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.996503 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.997242 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:41.997569 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:41.997629 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:41.999662 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:41.999670 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:41.999670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:41.999676 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:42.000960 (+ 1284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:42.001254 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:42.001256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:42.001256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:42.001309 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:42.001436 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:42.001436 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:42.001436 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:42.001552 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:42.001659 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:42.001711 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:47:42.001771 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":95,"threads_started":1}
Aug 09 08:47:42 dist-test-slave-ncp9 krb5kdc[16331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:42.008721 16455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:42.003074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53658 (local address 127.10.184.126:42619)
0809 08:47:42.003117 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:42.003118 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:42.003127 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:42.003193 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:42.003194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:42.003202 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:42.003247 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:42.003620 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:42.004107 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:42.004710 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:42.004880 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:42.004953 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:42.006788 (+ 1835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:42.006789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:42.006790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:42.006796 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:42.007884 (+ 1088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:42.008151 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:42.008152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:42.008152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:42.008195 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:42.008329 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:42.008329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:42.008330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:42.008455 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:42.008523 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:42.008588 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:47:42.008645 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 08:47:42.008971 16400 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53658
I20260809 08:47:42.009287 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 16495
I20260809 08:47:42.023590 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 16704
I20260809 08:47:42.036177 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 16905
I20260809 08:47:42.042939 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 16351
2026-08-09T08:47:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1971 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:42 dist-test-slave-ncp9 krb5kdc[17117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:42 dist-test-slave-ncp9 krb5kdc[17117](info): set up 2 sockets
Aug 09 08:47:42 dist-test-slave-ncp9 krb5kdc[17117](info): commencing operation
krb5kdc: starting...
W20260809 08:47:43.239540 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:43 dist-test-slave-ncp9 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:43.422079 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44183
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:43.591006 17141 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:43.591563 17141 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:43.591672 17141 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:43.593921 17141 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:43.593988 17141 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:43.594007 17141 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:43.594085 17141 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:43.594125 17141 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:43.596993 17141 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17141
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:43.597461 17141 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:43.597882 17141 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:43.606426 17150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:43.606449 17156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:43.606608 17151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:43.607048 17141 server_base.cc:1034] running on GCE node
I20260809 08:47:43.607497 17141 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:43.608107 17141 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:43.609305 17141 hybrid_clock.cc:648] HybridClock initialized: now 1786265263609269 us; error 50 us; skew 500 ppm
Aug 09 08:47:43 dist-test-slave-ncp9 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265263, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:43.613729 17141 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:43.614358 17141 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:43.614502 17141 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:43.614758 17141 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:43.616696 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "6cac2545722f4d81bdc072325a5b5bf7"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "7f5095f4971061cf8134a332ce84fc55"
server_key_iv: "9585be3a97ac0e4f991294879b161bba"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.617444 17141 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "6cac2545722f4d81bdc072325a5b5bf7"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "7f5095f4971061cf8134a332ce84fc55"
server_key_iv: "9585be3a97ac0e4f991294879b161bba"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.619956 17141 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:43.621309 17162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:43.621603 17141 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:43.621855 17141 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "6cac2545722f4d81bdc072325a5b5bf7"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "7f5095f4971061cf8134a332ce84fc55"
server_key_iv: "9585be3a97ac0e4f991294879b161bba"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.621994 17141 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:43.654086 17141 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:43.658056 17141 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:43.658392 17141 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:43.666808 17141 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44183
I20260809 08:47:43.666855 17237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44183 every 8 connection(s)
I20260809 08:47:43.667418 17141 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:43.669313 17238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:43.670295 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 17141
I20260809 08:47:43.670469 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:43.670877 10977 external_mini_cluster.cc:1428] Setting key 557abfdebd3a4be5ab1e8918e4aed67f
I20260809 08:47:43.675482 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: Bootstrap starting.
I20260809 08:47:43.677089 17238 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:43.677728 17238 log.cc:824] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:43.679117 17238 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: No bootstrap required, opened a new log
I20260809 08:47:43.683322 17238 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } }
I20260809 08:47:43.683712 17238 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:43.683794 17238 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cac2545722f4d81bdc072325a5b5bf7, State: Initialized, Role: FOLLOWER
Aug 09 08:47:43 dist-test-slave-ncp9 krb5kdc[17117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:43.684044 17238 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } }
I20260809 08:47:43.684227 17238 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:43.684319 17238 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:43.684422 17238 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:43.685799 17238 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } }
I20260809 08:47:43.685990 17238 leader_election.cc:302] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cac2545722f4d81bdc072325a5b5bf7; no voters:
I20260809 08:47:43.686265 17238 leader_election.cc:288] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:43.686322 17245 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:43.686566 17245 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [term 1 LEADER]: Becoming Leader. State: Replica: 6cac2545722f4d81bdc072325a5b5bf7, State: Running, Role: LEADER
I20260809 08:47:43.686753 17245 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } }
I20260809 08:47:43.686826 17238 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:43.687469 17245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cac2545722f4d81bdc072325a5b5bf7. Latest consensus state: current_term: 1 leader_uuid: "6cac2545722f4d81bdc072325a5b5bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } } }
I20260809 08:47:43.687595 17245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:43.688213 17247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cac2545722f4d81bdc072325a5b5bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cac2545722f4d81bdc072325a5b5bf7" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44183 } } }
I20260809 08:47:43.688347 17247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:43.689311 17254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:43.690606 17254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:43.692749 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:43.671755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59346 (local address 127.10.184.126:44183)
0809 08:47:43.672108 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:43.672129 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:43.672214 (+ 85us) server_negotiation.cc:407] Connection header received
0809 08:47:43.673454 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:43.673467 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:43.673606 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:43.673940 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:43.674945 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:43.677613 (+ 2668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:43.679131 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:43.680597 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:43.680838 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:43.685050 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:43.685064 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:43.685064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:43.685087 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:43.689382 (+ 4295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:43.690248 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:43.690252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:43.690253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:43.690444 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:43.690861 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:43.690862 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:43.690862 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:43.691508 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:43.691820 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:43.692088 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 08:47:43.692360 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":156,"threads_started":1}
I20260809 08:47:43.693794 17254 catalog_manager.cc:1347] Generated new cluster ID: 9a65539413174cf0a835f4f178af57b9
I20260809 08:47:43.693889 17254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:43.709479 17254 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:43.710777 17254 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:43.726176 17254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: Generated new TSK 0
I20260809 08:47:43.726672 17254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:43.777209 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:43.936094 17272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:43.936478 17272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:43.936569 17272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:43.938433 17272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:43.938501 17272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:43.938575 17272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:43.941538 17272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:43.942018 17272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:43.942415 17272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:43.953256 17284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:43.953266 17282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:43.954002 17281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:43.953899 17272 server_base.cc:1034] running on GCE node
I20260809 08:47:43.954589 17272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:43.955284 17272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:43.956555 17272 hybrid_clock.cc:648] HybridClock initialized: now 1786265263956505 us; error 63 us; skew 500 ppm
Aug 09 08:47:43 dist-test-slave-ncp9 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265263, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:43.962235 17272 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:43.962881 17272 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:43.962994 17272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:43.963217 17272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:43.965251 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "c029ea2cc7b2449aa57ad0b67984707c"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "c167e393b72313698b4af153b39cefec"
server_key_iv: "da71edf9d4513712c9a19701f6deea45"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.965960 17272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "c029ea2cc7b2449aa57ad0b67984707c"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "c167e393b72313698b4af153b39cefec"
server_key_iv: "da71edf9d4513712c9a19701f6deea45"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.968623 17272 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260809 08:47:43.970090 17298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:43.970428 17272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:43.970605 17272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "c029ea2cc7b2449aa57ad0b67984707c"
format_stamp: "Formatted at 2026-08-09 08:47:43 on dist-test-slave-ncp9"
server_key: "c167e393b72313698b4af153b39cefec"
server_key_iv: "da71edf9d4513712c9a19701f6deea45"
server_key_version: "encryptionkey@0"
I20260809 08:47:43.970734 17272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:44.016624 17272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:44.020282 17272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:44.020512 17272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:44.020967 17272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:44.021588 17272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:44.021651 17272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.021715 17272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:44.021781 17272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.035694 17272 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:35773
I20260809 08:47:44.035724 17469 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:35773 every 8 connection(s)
I20260809 08:47:44.036667 17272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:44.037277 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 17272
I20260809 08:47:44.037537 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:44.037840 10977 external_mini_cluster.cc:1428] Setting key eb4dc9b99d093943a160db7999b6c5c6
Aug 09 08:47:44 dist-test-slave-ncp9 krb5kdc[17117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265263, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
I20260809 08:47:44.061005 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.039992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:34313 (local address 127.10.184.126:44183)
0809 08:47:44.040101 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.040103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.040136 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:47:44.040591 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.040592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.040649 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.040762 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:44.043432 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.044724 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.047682 (+ 2958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.048352 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.048485 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.054229 (+ 5744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:44.054231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:44.054231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:44.054250 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:44.057684 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.058525 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.058526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.058527 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.058632 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.059280 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.059281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.059281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.059632 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:44.059746 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.060520 (+ 774us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.060795 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:47:44.061262 17472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.038709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44183 (local address 127.10.184.65:34313)
0809 08:47:44.039284 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.039351 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.040380 (+ 1029us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.041067 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.041082 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.041893 (+ 811us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:44.043210 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.043223 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.044929 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.044930 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.047510 (+ 2580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.047514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.048625 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.048626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.048974 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.050040 (+ 1066us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:44.050070 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:44.053975 (+ 3905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:44.057818 (+ 3843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.057834 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.057838 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.058364 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.058728 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.058729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.058729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.059097 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.059809 (+ 712us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:44.059826 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:44.060209 (+ 383us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.060403 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.060671 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":147,"threads_started":1}
I20260809 08:47:44.063510 17470 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44183
I20260809 08:47:44.063694 17470 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:44.063897 17470 heartbeater.cc:511] Master 127.10.184.126:44183 requested a full tablet report, sending...
I20260809 08:47:44.064883 17183 ts_manager.cc:194] Registered new tserver with Master: c029ea2cc7b2449aa57ad0b67984707c (127.10.184.65:35773)
I20260809 08:47:44.066002 17183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:34313
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:44.082374 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:44.234790 17477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:44.235203 17477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:44.235267 17477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:44.237187 17477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:44.237246 17477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:44.237318 17477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:44.239735 17477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:44.240293 17477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:44.240670 17477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:44.249361 17487 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:44.249357 17483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:44.249387 17484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:44.250198 17477 server_base.cc:1034] running on GCE node
I20260809 08:47:44.250597 17477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:44.251008 17477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:44.252262 17477 hybrid_clock.cc:648] HybridClock initialized: now 1786265264252230 us; error 56 us; skew 500 ppm
Aug 09 08:47:44 dist-test-slave-ncp9 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:44.256795 17477 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:44.257411 17477 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:44.257513 17477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:44.257701 17477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:44.259533 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "4f577b502b1545e9ad1d1cc526522392"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d153befb49b953bf1e61faa36e813e8c"
server_key_iv: "692f22a460646133a1db344e852624f2"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.260211 17477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "4f577b502b1545e9ad1d1cc526522392"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d153befb49b953bf1e61faa36e813e8c"
server_key_iv: "692f22a460646133a1db344e852624f2"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.262763 17477 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.005s
I20260809 08:47:44.263906 17496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.264214 17477 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:44.264348 17477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4f577b502b1545e9ad1d1cc526522392"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d153befb49b953bf1e61faa36e813e8c"
server_key_iv: "692f22a460646133a1db344e852624f2"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.264438 17477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:44.295696 17477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:44.299530 17477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:44.299738 17477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:44.300118 17477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:44.300686 17477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:44.300750 17477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.300818 17477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:44.300879 17477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.311772 17477 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:45067
I20260809 08:47:44.311877 17663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:45067 every 8 connection(s)
I20260809 08:47:44.312382 17477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:44.320752 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 17477
I20260809 08:47:44.320885 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:44.321262 10977 external_mini_cluster.cc:1428] Setting key fb7994d163937995344bd08944ab14a6
Aug 09 08:47:44 dist-test-slave-ncp9 krb5kdc[17117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:44.329001 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.313705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:40221 (local address 127.10.184.126:44183)
0809 08:47:44.313807 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.313811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.314159 (+ 348us) server_negotiation.cc:407] Connection header received
0809 08:47:44.315145 (+ 986us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.315147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.315184 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.315253 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:44.316984 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.317746 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.319405 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.319717 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.319806 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.323683 (+ 3877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:44.323685 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:44.323685 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:44.323704 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:44.326140 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.327054 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.327065 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.327065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.327159 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.327772 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.327773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.327773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.327958 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:44.328097 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.328635 (+ 538us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.328787 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 08:47:44.329349 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.313686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44183 (local address 127.10.184.66:40221)
0809 08:47:44.313987 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.314042 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.314968 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.315434 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.315439 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.315882 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:44.316831 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.316837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.317884 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.317884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.319318 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.319321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.319824 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.319825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.319999 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.320728 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:44.320745 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:44.323494 (+ 2749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:44.326250 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.326258 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.326268 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.326898 (+ 630us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.327272 (+ 374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.327273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.327274 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.327552 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.328139 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:44.328146 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:44.328367 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.328563 (+ 196us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.328834 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":180,"threads_started":1}
I20260809 08:47:44.330138 17665 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44183
I20260809 08:47:44.330269 17665 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:44.330503 17665 heartbeater.cc:511] Master 127.10.184.126:44183 requested a full tablet report, sending...
I20260809 08:47:44.331087 17183 ts_manager.cc:194] Registered new tserver with Master: 4f577b502b1545e9ad1d1cc526522392 (127.10.184.66:45067)
I20260809 08:47:44.331678 17183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:40221
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:44.359615 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:41897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:44.522405 17671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:44.522835 17671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:44.522899 17671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:44.524991 17671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:44.525066 17671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:44.525105 17671 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:44.527768 17671 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:41897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44183
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:44.528155 17671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:44.528611 17671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:44.537088 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:44.537014 17684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:44.537335 17671 server_base.cc:1034] running on GCE node
W20260809 08:47:44.537015 17683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:44.537909 17671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:44.538403 17671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:44.539629 17671 hybrid_clock.cc:648] HybridClock initialized: now 1786265264539609 us; error 60 us; skew 500 ppm
Aug 09 08:47:44 dist-test-slave-ncp9 krb5kdc[17117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:44.544204 17671 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:44.544735 17671 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:44.544934 17671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:44.545221 17671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:44.547232 17671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "b674c75af6694c9ab6e7e18e9a13dba6"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "e1541f8206d588b9fa7a1c305a44d3a3"
server_key_iv: "396e677e0b4df7336850d5e899bbbc0a"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.547861 17671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "b674c75af6694c9ab6e7e18e9a13dba6"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "e1541f8206d588b9fa7a1c305a44d3a3"
server_key_iv: "396e677e0b4df7336850d5e899bbbc0a"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.550099 17671 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.004s
I20260809 08:47:44.551335 17698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.551589 17671 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:44.551774 17671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "b674c75af6694c9ab6e7e18e9a13dba6"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "e1541f8206d588b9fa7a1c305a44d3a3"
server_key_iv: "396e677e0b4df7336850d5e899bbbc0a"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.551885 17671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:44.573803 17671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:44.577436 17671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:44.577831 17671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:44.578202 17671 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:44.578684 17671 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:44.578733 17671 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.578774 17671 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:44.578811 17671 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.593935 17671 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:37601
I20260809 08:47:44.593987 17866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:37601 every 8 connection(s)
I20260809 08:47:44.594353 17671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:44.597959 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 17671
I20260809 08:47:44.598201 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:44.598683 10977 external_mini_cluster.cc:1428] Setting key cb7e35a82cffa293d050361a706ef989
Aug 09 08:47:44 dist-test-slave-ncp9 krb5kdc[17117](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265264, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:44.611955 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.595488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48263 (local address 127.10.184.126:44183)
0809 08:47:44.595561 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.595563 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.595864 (+ 301us) server_negotiation.cc:407] Connection header received
0809 08:47:44.596658 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.596659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.596677 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.596750 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:44.599115 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.600381 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.602572 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.602963 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.603114 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.606960 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:44.606976 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:44.606977 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:44.606986 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:44.609253 (+ 2267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.610094 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.610095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.610095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.610192 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.610632 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.610632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.610633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.610898 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:44.611038 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.611580 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.611785 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:47:44.612278 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.595468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44183 (local address 127.10.184.67:48263)
0809 08:47:44.595690 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.595752 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.596525 (+ 773us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.596851 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.596856 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.597511 (+ 655us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:44.598952 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.598966 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.600577 (+ 1611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.600579 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.602430 (+ 1851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.602433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.603130 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.603132 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.603382 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.604197 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:44.604214 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:44.606825 (+ 2611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:44.609437 (+ 2612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.609457 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.609460 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.609944 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.610306 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:44.610306 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:44.610307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:44.610527 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:44.611026 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:44.611033 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:44.611264 (+ 231us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.611492 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.611856 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":98,"threads_started":1}
I20260809 08:47:44.613101 17867 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44183
I20260809 08:47:44.613317 17867 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:44.613655 17867 heartbeater.cc:511] Master 127.10.184.126:44183 requested a full tablet report, sending...
I20260809 08:47:44.614593 17183 ts_manager.cc:194] Registered new tserver with Master: b674c75af6694c9ab6e7e18e9a13dba6 (127.10.184.67:37601)
I20260809 08:47:44.615458 17183 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:48263
I20260809 08:47:44.622656 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:47:44.633852 17243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.624485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59352 (local address 127.10.184.126:44183)
0809 08:47:44.624572 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.624574 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.624686 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:47:44.624763 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.624764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.624813 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.624886 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:44.625763 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.626447 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.627443 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.627706 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.627778 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.629926 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:44.629927 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:44.629928 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:44.629934 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:44.632158 (+ 2224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.632707 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.632708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.632708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.632788 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:44.633123 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:44.633124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:44.633124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:44.633362 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:44.633491 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.633639 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.633741 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:47:44.636260 17183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:44.639173 17183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:44.650005 17885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.642875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:37601 (local address 127.0.0.1:53492)
0809 08:47:44.643500 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.643593 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.643702 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.644250 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.644253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.644262 (+ 9us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:44.644372 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.644374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.646089 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.646090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.647696 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.647699 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.649416 (+ 1717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.649418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.649581 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.649607 (+ 26us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.649806 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.649888 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":195,"threads_started":1}
I20260809 08:47:44.650012 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.643078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53492 (local address 127.10.184.67:37601)
0809 08:47:44.643944 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.643983 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.644003 (+ 20us) server_negotiation.cc:407] Connection header received
0809 08:47:44.644066 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.644068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.644097 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.644230 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:44.644510 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.645893 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.647891 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.649270 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.649420 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.649582 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.649820 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.649888 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":141,"threads_started":1}
I20260809 08:47:44.651134 17883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.642920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:35773 (local address 127.0.0.1:45370)
0809 08:47:44.643633 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.643650 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.643708 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.644350 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.644351 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.644361 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:44.644737 (+ 376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.644741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.647350 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.647351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.648983 (+ 1632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.648986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.650662 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.650663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.650794 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.650799 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.650934 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.651021 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":67,"thread_start_us":237,"threads_started":1}
I20260809 08:47:44.651757 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.643168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45370 (local address 127.10.184.65:35773)
0809 08:47:44.644005 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.644018 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.644049 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:47:44.644143 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.644147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.644194 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.644878 (+ 684us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:44.645034 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.647215 (+ 2181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.649117 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.650509 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.650656 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.650791 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.651531 (+ 740us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.651616 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":93,"threads_started":1}
I20260809 08:47:44.651732 17769 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff62998b2d74ce38d57f61c773b567a (DEFAULT_TABLE table=test-table [id=db4b61e5996e40c289a7a42a54d93e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:44.652585 17769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff62998b2d74ce38d57f61c773b567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:44.653175 17891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.643366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:45067 (local address 127.0.0.1:36466)
0809 08:47:44.644172 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.644191 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.644261 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.644846 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.644847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.644849 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:44.644958 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.644960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.647392 (+ 2432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.647393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.649218 (+ 1825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.649222 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.652549 (+ 3327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.652559 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.652752 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.652781 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.652932 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.653021 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":8,"thread_start_us":193,"threads_started":1}
I20260809 08:47:44.653002 17375 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff62998b2d74ce38d57f61c773b567a (DEFAULT_TABLE table=test-table [id=db4b61e5996e40c289a7a42a54d93e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:44.653213 17892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.643613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36466 (local address 127.10.184.66:45067)
0809 08:47:44.644063 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.644069 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.644368 (+ 299us) server_negotiation.cc:407] Connection header received
0809 08:47:44.644479 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.644498 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.644548 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.644702 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:44.645109 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.647215 (+ 2106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.649438 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.652416 (+ 2978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.652545 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.652684 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.652953 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.653052 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":138,"threads_started":1}
I20260809 08:47:44.653678 17375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff62998b2d74ce38d57f61c773b567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:44.657364 17894 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Bootstrap starting.
I20260809 08:47:44.658394 17895 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: Bootstrap starting.
I20260809 08:47:44.658424 17569 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff62998b2d74ce38d57f61c773b567a (DEFAULT_TABLE table=test-table [id=db4b61e5996e40c289a7a42a54d93e50]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:44.658990 17569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff62998b2d74ce38d57f61c773b567a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:44.659775 17895 tablet_bootstrap.cc:654] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:44.660285 17895 log.cc:824] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:44.661630 17895 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: No bootstrap required, opened a new log
I20260809 08:47:44.661827 17895 ts_tablet_manager.cc:1397] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:47:44.661923 17894 tablet_bootstrap.cc:654] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:44.663122 17894 log.cc:824] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:44.664953 17894 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: No bootstrap required, opened a new log
I20260809 08:47:44.665091 17894 ts_tablet_manager.cc:1397] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260809 08:47:44.664997 17899 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Bootstrap starting.
I20260809 08:47:44.666090 17899 tablet_bootstrap.cc:654] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:44.667224 17899 log.cc:824] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:44.667086 17895 raft_consensus.cc:348] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.667584 17895 raft_consensus.cc:374] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:44.667665 17895 raft_consensus.cc:729] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c029ea2cc7b2449aa57ad0b67984707c, State: Initialized, Role: FOLLOWER
I20260809 08:47:44.667979 17895 consensus_queue.cc:260] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.668159 17903 raft_consensus.cc:482] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:44.668280 17903 raft_consensus.cc:504] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.668955 17895 ts_tablet_manager.cc:1428] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 08:47:44.669200 17470 heartbeater.cc:503] Master 127.10.184.126:44183 was elected leader, sending a full tablet report...
I20260809 08:47:44.669442 17899 tablet_bootstrap.cc:492] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: No bootstrap required, opened a new log
I20260809 08:47:44.669147 17894 raft_consensus.cc:348] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.669529 17899 ts_tablet_manager.cc:1397] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260809 08:47:44.669569 17894 raft_consensus.cc:374] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:44.669660 17894 raft_consensus.cc:729] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b674c75af6694c9ab6e7e18e9a13dba6, State: Initialized, Role: FOLLOWER
I20260809 08:47:44.669922 17894 consensus_queue.cc:260] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.670739 17894 ts_tablet_manager.cc:1428] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 08:47:44.670917 17867 heartbeater.cc:503] Master 127.10.184.126:44183 was elected leader, sending a full tablet report...
I20260809 08:47:44.671195 17903 leader_election.cc:288] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b674c75af6694c9ab6e7e18e9a13dba6 (127.10.184.67:37601), 4f577b502b1545e9ad1d1cc526522392 (127.10.184.66:45067)
I20260809 08:47:44.674014 17899 raft_consensus.cc:348] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.674392 17899 raft_consensus.cc:374] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:44.674538 17899 raft_consensus.cc:729] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f577b502b1545e9ad1d1cc526522392, State: Initialized, Role: FOLLOWER
I20260809 08:47:44.674754 17899 consensus_queue.cc:260] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.675259 17665 heartbeater.cc:503] Master 127.10.184.126:44183 was elected leader, sending a full tablet report...
I20260809 08:47:44.675243 17899 ts_tablet_manager.cc:1428] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 08:47:44.678085 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.671722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:48365 (local address 127.10.184.67:37601)
0809 08:47:44.671789 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.671791 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.671994 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:47:44.672315 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.672324 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.672358 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.672470 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:44.672807 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.673960 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.676400 (+ 2440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.677355 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.677533 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.677560 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.677911 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.677992 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:47:44.678404 17907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.671412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:37601 (local address 127.10.184.65:48365)
0809 08:47:44.671883 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.671903 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.672047 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.672505 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.672506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.672517 (+ 11us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:44.672639 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.672656 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.674100 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.674108 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.676230 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.676242 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.677520 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.677527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.677643 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.677649 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.678192 (+ 543us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.678260 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":148,"threads_started":1}
I20260809 08:47:44.679044 17796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff62998b2d74ce38d57f61c773b567a" candidate_uuid: "c029ea2cc7b2449aa57ad0b67984707c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" is_pre_election: true
I20260809 08:47:44.679327 17796 raft_consensus.cc:2454] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c029ea2cc7b2449aa57ad0b67984707c in term 0.
I20260809 08:47:44.679479 17892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.671799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:59009 (local address 127.10.184.66:45067)
0809 08:47:44.671959 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.671962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.673640 (+ 1678us) server_negotiation.cc:407] Connection header received
0809 08:47:44.673806 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.673807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.673824 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.673895 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:44.674291 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.675355 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.677893 (+ 2538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.678730 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.678934 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.678944 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.679280 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.679386 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260809 08:47:44.679520 17908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.671414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:45067 (local address 127.10.184.65:59009)
0809 08:47:44.673546 (+ 2132us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.673565 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:44.673681 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:44.674056 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:44.674056 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:44.674058 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:44.674176 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.674178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.675521 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.675522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.677540 (+ 2018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:44.677544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.679006 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.679016 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:44.679136 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.679140 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:44.679262 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:47:44.679376 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2106,"mutex_wait_us":19,"thread_start_us":152,"threads_started":1}
I20260809 08:47:44.679806 17302 leader_election.cc:302] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b674c75af6694c9ab6e7e18e9a13dba6, c029ea2cc7b2449aa57ad0b67984707c; no voters:
I20260809 08:47:44.680030 17904 raft_consensus.cc:2781] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:44.680133 17904 raft_consensus.cc:482] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:44.680225 17904 raft_consensus.cc:3037] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:44.680227 17595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff62998b2d74ce38d57f61c773b567a" candidate_uuid: "c029ea2cc7b2449aa57ad0b67984707c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f577b502b1545e9ad1d1cc526522392" is_pre_election: true
I20260809 08:47:44.680598 17595 raft_consensus.cc:2454] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c029ea2cc7b2449aa57ad0b67984707c in term 0.
I20260809 08:47:44.681632 17904 raft_consensus.cc:504] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.681887 17904 leader_election.cc:288] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [CANDIDATE]: Term 1 election: Requested vote from peers b674c75af6694c9ab6e7e18e9a13dba6 (127.10.184.67:37601), 4f577b502b1545e9ad1d1cc526522392 (127.10.184.66:45067)
I20260809 08:47:44.681982 17595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff62998b2d74ce38d57f61c773b567a" candidate_uuid: "c029ea2cc7b2449aa57ad0b67984707c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f577b502b1545e9ad1d1cc526522392"
I20260809 08:47:44.682000 17796 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff62998b2d74ce38d57f61c773b567a" candidate_uuid: "c029ea2cc7b2449aa57ad0b67984707c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b674c75af6694c9ab6e7e18e9a13dba6"
I20260809 08:47:44.682106 17796 raft_consensus.cc:3037] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:44.682124 17595 raft_consensus.cc:3037] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:44.683147 17796 raft_consensus.cc:2454] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c029ea2cc7b2449aa57ad0b67984707c in term 1.
I20260809 08:47:44.683248 17595 raft_consensus.cc:2454] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c029ea2cc7b2449aa57ad0b67984707c in term 1.
I20260809 08:47:44.683359 17302 leader_election.cc:302] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b674c75af6694c9ab6e7e18e9a13dba6, c029ea2cc7b2449aa57ad0b67984707c; no voters:
I20260809 08:47:44.683518 17904 raft_consensus.cc:2781] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:44.683629 17904 raft_consensus.cc:686] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 1 LEADER]: Becoming Leader. State: Replica: c029ea2cc7b2449aa57ad0b67984707c, State: Running, Role: LEADER
I20260809 08:47:44.683818 17904 consensus_queue.cc:237] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } }
I20260809 08:47:44.685187 17183 catalog_manager.cc:5495] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c reported cstate change: term changed from 0 to 1, leader changed from <none> to c029ea2cc7b2449aa57ad0b67984707c (127.10.184.65). New cstate: current_term: 1 leader_uuid: "c029ea2cc7b2449aa57ad0b67984707c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c029ea2cc7b2449aa57ad0b67984707c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35773 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:44.696517 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:44.693135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45382 (local address 127.10.184.65:35773)
0809 08:47:44.693211 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:44.693212 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:44.693228 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:47:44.693264 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:44.693270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:44.693296 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:44.693388 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:44.693614 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.694304 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.695547 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:44.695832 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:44.695907 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:44.696098 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:44.696256 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:44.696342 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:44.696372 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 08:47:44.696396 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:47:44.699550 17595 raft_consensus.cc:1261] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 1 FOLLOWER]: Refusing update from remote peer c029ea2cc7b2449aa57ad0b67984707c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:44.699561 17796 raft_consensus.cc:1261] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 1 FOLLOWER]: Refusing update from remote peer c029ea2cc7b2449aa57ad0b67984707c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:44.700173 17903 consensus_queue.cc:1035] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b674c75af6694c9ab6e7e18e9a13dba6" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37601 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:44.700336 17904 consensus_queue.cc:1035] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f577b502b1545e9ad1d1cc526522392" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 45067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:44.705061 17915 mvcc.cc:205] Tried to move back new op lower bound from 7316542524205150208 to 7316542524146126848. Current Snapshot: MvccSnapshot[applied={T|T < 7316542524205150208}]
I20260809 08:47:44.705848 17917 mvcc.cc:205] Tried to move back new op lower bound from 7316542524205150208 to 7316542524146126848. Current Snapshot: MvccSnapshot[applied={T|T < 7316542524205150208}]
I20260809 08:47:44.708314 17918 mvcc.cc:205] Tried to move back new op lower bound from 7316542524205150208 to 7316542524146126848. Current Snapshot: MvccSnapshot[applied={T|T < 7316542524205150208}]
I20260809 08:47:44.709918 17181 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:44.710103 17181 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:44.711374 17181 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6cac2545722f4d81bdc072325a5b5bf7: Sending DeleteTablet for 3 replicas of tablet 4ff62998b2d74ce38d57f61c773b567a
I20260809 08:47:44.711956 17769 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff62998b2d74ce38d57f61c773b567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:44 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:53492
I20260809 08:47:44.712067 17375 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff62998b2d74ce38d57f61c773b567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:44 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:45370
I20260809 08:47:44.712360 17936 tablet_replica.cc:330] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: stopping tablet replica
I20260809 08:47:44.712383 17937 tablet_replica.cc:330] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c: stopping tablet replica
I20260809 08:47:44.712407 17569 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ff62998b2d74ce38d57f61c773b567a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:44 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36466
I20260809 08:47:44.712548 17936 raft_consensus.cc:2229] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:44.712575 17937 raft_consensus.cc:2229] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:44.712779 17937 raft_consensus.cc:2258] T 4ff62998b2d74ce38d57f61c773b567a P c029ea2cc7b2449aa57ad0b67984707c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:44.712787 17936 raft_consensus.cc:2258] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:44.712996 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 17272
I20260809 08:47:44.730760 17938 tablet_replica.cc:330] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: stopping tablet replica
I20260809 08:47:44.731106 17938 raft_consensus.cc:2229] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:44.731508 17938 raft_consensus.cc:2258] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:44.731910 17936 ts_tablet_manager.cc:1905] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:44.732545 17938 ts_tablet_manager.cc:1905] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:44.735357 17936 ts_tablet_manager.cc:1918] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:44.735498 17936 log.cc:1196] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/4ff62998b2d74ce38d57f61c773b567a
I20260809 08:47:44.735970 17938 ts_tablet_manager.cc:1918] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:44.736083 17936 ts_tablet_manager.cc:1939] T 4ff62998b2d74ce38d57f61c773b567a P b674c75af6694c9ab6e7e18e9a13dba6: Deleting consensus metadata
I20260809 08:47:44.736266 17938 log.cc:1196] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/4ff62998b2d74ce38d57f61c773b567a
I20260809 08:47:44.736832 17938 ts_tablet_manager.cc:1939] T 4ff62998b2d74ce38d57f61c773b567a P 4f577b502b1545e9ad1d1cc526522392: Deleting consensus metadata
I20260809 08:47:44.736950 17165 catalog_manager.cc:4841] TS b674c75af6694c9ab6e7e18e9a13dba6 (127.10.184.67:37601): tablet 4ff62998b2d74ce38d57f61c773b567a (table test-table [id=db4b61e5996e40c289a7a42a54d93e50]) successfully deleted
W20260809 08:47:44.737142 17163 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:35773 (error 108)
I20260809 08:47:44.737594 17163 catalog_manager.cc:4841] TS 4f577b502b1545e9ad1d1cc526522392 (127.10.184.66:45067): tablet 4ff62998b2d74ce38d57f61c773b567a (table test-table [id=db4b61e5996e40c289a7a42a54d93e50]) successfully deleted
W20260809 08:47:44.738035 17891 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:44.737854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:35773 (local address 127.0.0.1:45392)
0809 08:47:44.737914 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:44.737972 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:35773: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 08:47:44.738282 17163 catalog_manager.cc:4584] TS c029ea2cc7b2449aa57ad0b67984707c (127.10.184.65:35773): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ff62998b2d74ce38d57f61c773b567a: Network error: Client connection negotiation failed: client connection to 127.10.184.65:35773: connect: Connection reset by peer (error 104)
I20260809 08:47:44.738372 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 17477
I20260809 08:47:44.755695 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 17671
I20260809 08:47:44.769557 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 17141
2026-08-09T08:47:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2736 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T08:47:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:44Z Disabled control of system clock
I20260809 08:47:44.817871 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40485
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:47:44.972056 17943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:44.972489 17943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:44.972520 17943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:44.974258 17943 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:44.974323 17943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:44.974371 17943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:44.974408 17943 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:44.974442 17943 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:44.976622 17943 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:40485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:44.977026 17943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:44.977573 17943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:44.985669 17957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:44.985687 17955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:44.985675 17954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:44.986241 17943 server_base.cc:1034] running on GCE node
I20260809 08:47:44.986677 17943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:44.987181 17943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:44.988368 17943 hybrid_clock.cc:648] HybridClock initialized: now 1786265264988327 us; error 55 us; skew 500 ppm
I20260809 08:47:44.990547 17943 webserver.cc:460] Webserver started at http://127.10.184.126:35981/ using document root <none> and password file <none>
I20260809 08:47:44.991025 17943 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:44.991124 17943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:44.991384 17943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:44.993378 17943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "acbad63817644c8db7ddf6ac1d9eacb3"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d179cc93ce176bf316cfec8970f0b7eb"
server_key_iv: "1b8a1e9f68ffc58411a25c437f5ac4d1"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.994091 17943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "acbad63817644c8db7ddf6ac1d9eacb3"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d179cc93ce176bf316cfec8970f0b7eb"
server_key_iv: "1b8a1e9f68ffc58411a25c437f5ac4d1"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.996302 17943 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260809 08:47:44.997545 17965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:44.997745 17943 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:47:44.997877 17943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "acbad63817644c8db7ddf6ac1d9eacb3"
format_stamp: "Formatted at 2026-08-09 08:47:44 on dist-test-slave-ncp9"
server_key: "d179cc93ce176bf316cfec8970f0b7eb"
server_key_iv: "1b8a1e9f68ffc58411a25c437f5ac4d1"
server_key_version: "encryptionkey@0"
I20260809 08:47:44.997989 17943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:45.005314 17943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:45.005812 17943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:45.006081 17943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:45.014065 17943 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:40485
I20260809 08:47:45.014117 18050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:40485 every 8 connection(s)
I20260809 08:47:45.014704 17943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:45.015924 18053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:45.021021 18053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: Bootstrap starting.
I20260809 08:47:45.022007 18053 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:45.022550 18053 log.cc:824] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:45.023651 18053 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: No bootstrap required, opened a new log
I20260809 08:47:45.024782 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 17943
I20260809 08:47:45.025007 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:45.025375 10977 external_mini_cluster.cc:1428] Setting key fb53e6b9e43d41d93ce5c6a35ada9dc1
I20260809 08:47:45.027638 18053 raft_consensus.cc:348] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } }
I20260809 08:47:45.027999 18053 raft_consensus.cc:374] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:45.028098 18053 raft_consensus.cc:729] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acbad63817644c8db7ddf6ac1d9eacb3, State: Initialized, Role: FOLLOWER
I20260809 08:47:45.028337 18053 consensus_queue.cc:260] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } }
I20260809 08:47:45.028447 18053 raft_consensus.cc:388] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:45.028486 18053 raft_consensus.cc:482] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:45.028600 18053 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:45.029834 18053 raft_consensus.cc:504] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } }
I20260809 08:47:45.030004 18053 leader_election.cc:302] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acbad63817644c8db7ddf6ac1d9eacb3; no voters:
I20260809 08:47:45.030115 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.026356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51142 (local address 127.10.184.126:40485)
0809 08:47:45.026626 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.026642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.026694 (+ 52us) server_negotiation.cc:407] Connection header received
0809 08:47:45.027568 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.027577 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.027657 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.027871 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.029142 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.029162 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.029163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.029185 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.029285 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.029305 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.029394 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.029569 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.029670 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":103,"threads_started":1}
I20260809 08:47:45.030251 18053 leader_election.cc:288] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:45.030845 18053 sys_catalog.cc:564] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:45.030885 18059 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:45.031137 18059 raft_consensus.cc:686] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [term 1 LEADER]: Becoming Leader. State: Replica: acbad63817644c8db7ddf6ac1d9eacb3, State: Running, Role: LEADER
I20260809 08:47:45.031378 18059 consensus_queue.cc:237] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } }
I20260809 08:47:45.032092 18063 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acbad63817644c8db7ddf6ac1d9eacb3. Latest consensus state: current_term: 1 leader_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } } }
I20260809 08:47:45.032282 18063 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:45.032455 18061 sys_catalog.cc:455] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acbad63817644c8db7ddf6ac1d9eacb3" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40485 } } }
I20260809 08:47:45.032629 18061 sys_catalog.cc:458] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:45.032867 18072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:45.034369 18072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:45.037143 18072 catalog_manager.cc:1347] Generated new cluster ID: 345942f0098c40ed964be84e9b39fe37
I20260809 08:47:45.037220 18072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:45.054230 18072 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:45.054512 18072 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:45.059815 18072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: Generated new TSK 0
I20260809 08:47:45.060166 18072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:45.071234 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--builtin_ntp_servers=127.10.184.84:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:47:45.233593 18085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:45.233961 18085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:45.234035 18085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:45.235872 18085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:45.235944 18085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:45.236030 18085 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:45.238198 18085 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:45.238639 18085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:45.239149 18085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:45.249265 18092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.249599 18094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.249782 18091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:45.249797 18085 server_base.cc:1034] running on GCE node
I20260809 08:47:45.250221 18085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:45.250672 18085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:45.251899 18085 hybrid_clock.cc:648] HybridClock initialized: now 1786265265251876 us; error 47 us; skew 500 ppm
I20260809 08:47:45.254012 18085 webserver.cc:460] Webserver started at http://127.10.184.65:41231/ using document root <none> and password file <none>
I20260809 08:47:45.254449 18085 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:45.254555 18085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:45.254812 18085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:45.256722 18085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "d4abd97fd5dd4c5cb98978b44ac9e932"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "d5a3224c2aae573ccc9f724cca8e4a4f"
server_key_iv: "97917bf96fb95e124dd2b2eaff69cdec"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.257560 18085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "d4abd97fd5dd4c5cb98978b44ac9e932"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "d5a3224c2aae573ccc9f724cca8e4a4f"
server_key_iv: "97917bf96fb95e124dd2b2eaff69cdec"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.260087 18085 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:45.261515 18101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.261758 18085 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:47:45.261894 18085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "d4abd97fd5dd4c5cb98978b44ac9e932"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "d5a3224c2aae573ccc9f724cca8e4a4f"
server_key_iv: "97917bf96fb95e124dd2b2eaff69cdec"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.262022 18085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:45.268690 18085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:45.269182 18085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:45.269428 18085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:45.269796 18085 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:45.270298 18085 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:45.270359 18085 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.270419 18085 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:45.270466 18085 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.281605 18085 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:46241
I20260809 08:47:45.281682 18280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:46241 every 8 connection(s)
I20260809 08:47:45.282228 18085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:45.288298 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.283537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:60583 (local address 127.10.184.126:40485)
0809 08:47:45.283622 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.283624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.284202 (+ 578us) server_negotiation.cc:407] Connection header received
0809 08:47:45.285478 (+ 1276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.285479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.285487 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.285566 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.287924 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.287926 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.287926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.287938 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.287953 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.287958 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.288018 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.288182 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.288196 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:47:45.288657 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.283554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:40485 (local address 127.10.184.65:60583)
0809 08:47:45.283927 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.284050 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.285292 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.285753 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.285757 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.285800 (+ 43us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.287656 (+ 1856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.287656 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.287726 (+ 70us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.287793 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.288032 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.288042 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.288046 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.288125 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.288249 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":142,"threads_started":1}
I20260809 08:47:45.289492 18281 heartbeater.cc:347] Connected to a master server at 127.10.184.126:40485
I20260809 08:47:45.289644 18281 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:45.289969 18281 heartbeater.cc:511] Master 127.10.184.126:40485 requested a full tablet report, sending...
I20260809 08:47:45.290817 17988 ts_manager.cc:194] Registered new tserver with Master: d4abd97fd5dd4c5cb98978b44ac9e932 (127.10.184.65:46241)
I20260809 08:47:45.291419 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 18085
I20260809 08:47:45.291623 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:45.291996 10977 external_mini_cluster.cc:1428] Setting key ff89086600847d16e6b55866e0a46065
I20260809 08:47:45.293687 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--builtin_ntp_servers=127.10.184.84:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:47:45.443001 18284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:45.443472 18284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:45.443539 18284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:45.446110 18284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:45.446187 18284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:45.446245 18284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:45.449448 18284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:45.449911 18284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:45.450481 18284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:45.460239 18293 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.460271 18292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:45.460321 18284 server_base.cc:1034] running on GCE node
W20260809 08:47:45.460258 18296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:45.460834 18284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:45.461334 18284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:45.462507 18284 hybrid_clock.cc:648] HybridClock initialized: now 1786265265462478 us; error 44 us; skew 500 ppm
I20260809 08:47:45.464469 18284 webserver.cc:460] Webserver started at http://127.10.184.66:36803/ using document root <none> and password file <none>
I20260809 08:47:45.464865 18284 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:45.465037 18284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:45.465291 18284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:45.467244 18284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "5431df10cbf74528b9349d9c6625f73b"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "34087de0a82982cb619cf732e25e57fb"
server_key_iv: "a1a72a95388ffde46404f6a476af9bc9"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.467926 18284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "5431df10cbf74528b9349d9c6625f73b"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "34087de0a82982cb619cf732e25e57fb"
server_key_iv: "a1a72a95388ffde46404f6a476af9bc9"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.470703 18284 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.003s
I20260809 08:47:45.472019 18304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.472275 18284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:45.472430 18284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "5431df10cbf74528b9349d9c6625f73b"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "34087de0a82982cb619cf732e25e57fb"
server_key_iv: "a1a72a95388ffde46404f6a476af9bc9"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.472543 18284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:45.479146 18284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:45.479626 18284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:45.479897 18284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:45.480324 18284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:45.480880 18284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:45.480975 18284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.481045 18284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:45.481068 18284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.491739 18284 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:40799
I20260809 08:47:45.491840 18476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:40799 every 8 connection(s)
I20260809 08:47:45.492436 18284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:45.498005 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.493694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:37617 (local address 127.10.184.126:40485)
0809 08:47:45.493787 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.493795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.494238 (+ 443us) server_negotiation.cc:407] Connection header received
0809 08:47:45.495217 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.495219 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.495244 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.495364 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.497537 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.497538 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.497539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.497544 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.497575 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.497581 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.497682 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.497840 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.497876 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:47:45.498430 18479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.493699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:40485 (local address 127.10.184.66:37617)
0809 08:47:45.493997 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.494038 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.495023 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.495527 (+ 504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.495531 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.495539 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.497244 (+ 1705us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.497244 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.497306 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.497392 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.497679 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.497684 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.497695 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.497828 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.497956 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":128,"threads_started":1}
I20260809 08:47:45.499166 18477 heartbeater.cc:347] Connected to a master server at 127.10.184.126:40485
I20260809 08:47:45.499303 18477 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:45.499538 18477 heartbeater.cc:511] Master 127.10.184.126:40485 requested a full tablet report, sending...
I20260809 08:47:45.500105 17988 ts_manager.cc:194] Registered new tserver with Master: 5431df10cbf74528b9349d9c6625f73b (127.10.184.66:40799)
I20260809 08:47:45.502059 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 18284
I20260809 08:47:45.502238 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:45.502532 10977 external_mini_cluster.cc:1428] Setting key 1e2257ca8203a8e14bb6dd18c8747dd1
I20260809 08:47:45.503904 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--builtin_ntp_servers=127.10.184.84:46273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 08:47:45.604674 18481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:45.604871 18481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:45.604913 18481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:45.606112 18481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:45.606150 18481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:45.606174 18481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:45.607430 18481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:46273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:45.607611 18481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:45.607844 18481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:45.612509 18491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.612514 18493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.612521 18490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:45.612821 18481 server_base.cc:1034] running on GCE node
I20260809 08:47:45.613029 18481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:45.613236 18481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:45.614360 18481 hybrid_clock.cc:648] HybridClock initialized: now 1786265265614351 us; error 26 us; skew 500 ppm
I20260809 08:47:45.615499 18481 webserver.cc:460] Webserver started at http://127.10.184.67:41359/ using document root <none> and password file <none>
I20260809 08:47:45.615710 18481 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:45.615756 18481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:45.615881 18481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:45.617038 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "4cd416f1399e4f8f9d3d0b9681f5b791"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "6c5a198e0ffed03fcf5fa7ca7c660f12"
server_key_iv: "fc632e8a495d9112aee87b7e5712b086"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.617408 18481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "4cd416f1399e4f8f9d3d0b9681f5b791"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "6c5a198e0ffed03fcf5fa7ca7c660f12"
server_key_iv: "fc632e8a495d9112aee87b7e5712b086"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.618809 18481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:47:45.619445 18503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.619601 18481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.619692 18481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "4cd416f1399e4f8f9d3d0b9681f5b791"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "6c5a198e0ffed03fcf5fa7ca7c660f12"
server_key_iv: "fc632e8a495d9112aee87b7e5712b086"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.619750 18481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:45.622960 18481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:45.623179 18481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:45.623289 18481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:45.623461 18481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:45.623744 18481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:45.623777 18481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.623802 18481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:45.623826 18481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.628995 18481 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:39139
I20260809 08:47:45.629046 18680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:39139 every 8 connection(s)
I20260809 08:47:45.629313 18481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:45.631126 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 18481
I20260809 08:47:45.631289 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:45.631489 10977 external_mini_cluster.cc:1428] Setting key 467033a425d4fa15e5758de0564c2538
I20260809 08:47:45.632272 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.629957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:50649 (local address 127.10.184.126:40485)
0809 08:47:45.629998 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.630000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.630214 (+ 214us) server_negotiation.cc:407] Connection header received
0809 08:47:45.630627 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.630627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.630632 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.630693 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.631969 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.631970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.631970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.631974 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.631989 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.631993 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.632071 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.632161 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.632203 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:45.632436 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.629936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:40485 (local address 127.10.184.67:50649)
0809 08:47:45.630112 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.630149 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.630553 (+ 404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.630742 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.630745 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.630753 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.631796 (+ 1043us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.631797 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.631834 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.631869 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.632072 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.632077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.632080 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.632186 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.632247 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
I20260809 08:47:45.632882 18681 heartbeater.cc:347] Connected to a master server at 127.10.184.126:40485
I20260809 08:47:45.632992 18681 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:45.633131 18681 heartbeater.cc:511] Master 127.10.184.126:40485 requested a full tablet report, sending...
I20260809 08:47:45.633433 17988 ts_manager.cc:194] Registered new tserver with Master: 4cd416f1399e4f8f9d3d0b9681f5b791 (127.10.184.67:39139)
I20260809 08:47:45.642946 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:47:45.644924 18058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.644086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51154 (local address 127.10.184.126:40485)
0809 08:47:45.644153 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.644155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.644196 (+ 41us) server_negotiation.cc:407] Connection header received
0809 08:47:45.644246 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.644247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.644250 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.644293 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.644738 (+ 445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.644739 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.644739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.644743 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.644755 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.644758 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.644799 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.644829 (+ 30us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.644838 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 08:47:45.646378 17988 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:45.647830 17988 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:45.651486 18699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:40799 (local address 127.0.0.1:58448)
0809 08:47:45.650139 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.650181 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.650236 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.650586 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.650594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.650600 (+ 6us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.651078 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.651079 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.651096 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.651119 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.651355 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.651362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.651366 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.651409 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.651423 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":60,"threads_started":1}
I20260809 08:47:45.651636 18696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39139 (local address 127.0.0.1:52234)
0809 08:47:45.650443 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.650453 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.650492 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.650744 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.650745 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.650747 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.651274 (+ 527us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.651275 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.651280 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.651290 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.651513 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.651513 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.651514 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.651557 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.651570 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":93,"threads_started":1}
I20260809 08:47:45.651652 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52234 (local address 127.10.184.67:39139)
0809 08:47:45.650608 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.650612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.650621 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:45.650641 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.650643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.650650 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.650872 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.651366 (+ 494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.651374 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.651374 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.651379 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.651413 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.651436 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.651516 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.651562 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.651576 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":77,"threads_started":1}
I20260809 08:47:45.651819 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58448 (local address 127.10.184.66:40799)
0809 08:47:45.650435 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.650439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.650450 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:45.650477 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.650479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.650489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.650821 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.651202 (+ 381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.651208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.651209 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.651213 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.651248 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.651271 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.651692 (+ 421us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.651719 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.651737 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260809 08:47:45.652266 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:46241 (local address 127.0.0.1:37336)
0809 08:47:45.650179 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.650188 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.650237 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.650629 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.650630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.650633 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.651392 (+ 759us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.651393 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.651398 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.651408 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.652136 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.652137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.652138 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.652195 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.652207 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":4,"thread_start_us":73,"threads_started":1}
I20260809 08:47:45.652295 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.650289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.10.184.65:46241)
0809 08:47:45.650452 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.650455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.650465 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:45.650490 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.650492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.650501 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.651135 (+ 634us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.651485 (+ 350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.651489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.651490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.651493 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.651862 (+ 369us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.652061 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.652128 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.652202 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.652217 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260809 08:47:45.652319 18587 tablet_service.cc:1467] Processing CreateTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 (DEFAULT_TABLE table=test-table [id=fedc0ca30eaf45c4a06e53ab5b532550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:45.652446 18386 tablet_service.cc:1467] Processing CreateTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 (DEFAULT_TABLE table=test-table [id=fedc0ca30eaf45c4a06e53ab5b532550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:45.652709 18587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898ea0a15774ddd8c1cb2016f3d4685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:45.652814 18386 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898ea0a15774ddd8c1cb2016f3d4685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:45.653216 18174 tablet_service.cc:1467] Processing CreateTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 (DEFAULT_TABLE table=test-table [id=fedc0ca30eaf45c4a06e53ab5b532550]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:45.653535 18174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5898ea0a15774ddd8c1cb2016f3d4685. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:45.655381 18708 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Bootstrap starting.
I20260809 08:47:45.655965 18709 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Bootstrap starting.
I20260809 08:47:45.656167 18708 tablet_bootstrap.cc:654] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:45.656491 18708 log.cc:824] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:45.656682 18709 tablet_bootstrap.cc:654] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:45.656659 18710 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: Bootstrap starting.
I20260809 08:47:45.657028 18709 log.cc:824] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:45.657307 18708 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: No bootstrap required, opened a new log
I20260809 08:47:45.657380 18708 ts_tablet_manager.cc:1397] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:45.657493 18710 tablet_bootstrap.cc:654] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:45.657701 18709 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: No bootstrap required, opened a new log
I20260809 08:47:45.657759 18709 ts_tablet_manager.cc:1397] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:45.657807 18710 log.cc:824] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:45.658608 18710 tablet_bootstrap.cc:492] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: No bootstrap required, opened a new log
I20260809 08:47:45.658689 18710 ts_tablet_manager.cc:1397] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:45.659651 18709 raft_consensus.cc:348] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.659781 18709 raft_consensus.cc:374] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:45.659812 18709 raft_consensus.cc:729] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5431df10cbf74528b9349d9c6625f73b, State: Initialized, Role: FOLLOWER
I20260809 08:47:45.659812 18708 raft_consensus.cc:348] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.659973 18708 raft_consensus.cc:374] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:45.659942 18709 consensus_queue.cc:260] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.660015 18708 raft_consensus.cc:729] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cd416f1399e4f8f9d3d0b9681f5b791, State: Initialized, Role: FOLLOWER
I20260809 08:47:45.660207 18709 ts_tablet_manager.cc:1428] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:47:45.660202 18708 consensus_queue.cc:260] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.660320 18477 heartbeater.cc:503] Master 127.10.184.126:40485 was elected leader, sending a full tablet report...
I20260809 08:47:45.660450 18708 ts_tablet_manager.cc:1428] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:47:45.660576 18681 heartbeater.cc:503] Master 127.10.184.126:40485 was elected leader, sending a full tablet report...
I20260809 08:47:45.661094 18710 raft_consensus.cc:348] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.661208 18710 raft_consensus.cc:374] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:45.661239 18710 raft_consensus.cc:729] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4abd97fd5dd4c5cb98978b44ac9e932, State: Initialized, Role: FOLLOWER
I20260809 08:47:45.661342 18710 consensus_queue.cc:260] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.661599 18281 heartbeater.cc:503] Master 127.10.184.126:40485 was elected leader, sending a full tablet report...
I20260809 08:47:45.661794 18710 ts_tablet_manager.cc:1428] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:47:45.698853 18721 raft_consensus.cc:482] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:45.698976 18721 raft_consensus.cc:504] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.699285 18721 leader_election.cc:288] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5431df10cbf74528b9349d9c6625f73b (127.10.184.66:40799), 4cd416f1399e4f8f9d3d0b9681f5b791 (127.10.184.67:39139)
I20260809 08:47:45.700697 18703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.699733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40467 (local address 127.10.184.66:40799)
0809 08:47:45.699802 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.699805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.699812 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:45.699834 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.699835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.699839 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.699890 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.700487 (+ 597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.700488 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.700489 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.700492 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.700507 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.700511 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.700563 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.700608 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.700621 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:47:45.700691 18705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.699653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:50371 (local address 127.10.184.67:39139)
0809 08:47:45.699701 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.699704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.699732 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:47:45.699802 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.699803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.699809 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.699855 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.700464 (+ 609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.700466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.700466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.700471 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.700491 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.700496 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.700556 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.700607 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.700621 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:47:45.700701 18283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.699462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:40799 (local address 127.10.184.65:40467)
0809 08:47:45.699525 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.699537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.699573 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.699908 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.699909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.699911 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.700398 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.700399 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.700404 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.700412 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.700563 (+ 151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.700564 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.700564 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.700605 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.700614 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:47:45.700712 18727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.699464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39139 (local address 127.10.184.65:50371)
0809 08:47:45.699674 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.699685 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.699734 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.699866 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.699866 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.699869 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:45.700351 (+ 482us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:47:45.700351 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:45.700359 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:47:45.700374 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:45.700554 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:45.700555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:45.700556 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:45.700598 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:45.700612 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":14,"thread_start_us":103,"threads_started":1}
I20260809 08:47:45.701056 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898ea0a15774ddd8c1cb2016f3d4685" candidate_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" is_pre_election: true
I20260809 08:47:45.701081 18414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898ea0a15774ddd8c1cb2016f3d4685" candidate_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5431df10cbf74528b9349d9c6625f73b" is_pre_election: true
I20260809 08:47:45.701207 18618 raft_consensus.cc:2454] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4abd97fd5dd4c5cb98978b44ac9e932 in term 0.
I20260809 08:47:45.701207 18414 raft_consensus.cc:2454] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4abd97fd5dd4c5cb98978b44ac9e932 in term 0.
I20260809 08:47:45.701432 18103 leader_election.cc:302] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd416f1399e4f8f9d3d0b9681f5b791, d4abd97fd5dd4c5cb98978b44ac9e932; no voters:
I20260809 08:47:45.701599 18721 raft_consensus.cc:2781] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:45.701637 18721 raft_consensus.cc:482] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:45.701659 18721 raft_consensus.cc:3037] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:45.702355 18721 raft_consensus.cc:504] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.702483 18721 leader_election.cc:288] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [CANDIDATE]: Term 1 election: Requested vote from peers 5431df10cbf74528b9349d9c6625f73b (127.10.184.66:40799), 4cd416f1399e4f8f9d3d0b9681f5b791 (127.10.184.67:39139)
I20260809 08:47:45.702621 18414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898ea0a15774ddd8c1cb2016f3d4685" candidate_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5431df10cbf74528b9349d9c6625f73b"
I20260809 08:47:45.702637 18618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5898ea0a15774ddd8c1cb2016f3d4685" candidate_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791"
I20260809 08:47:45.702700 18618 raft_consensus.cc:3037] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:45.702706 18414 raft_consensus.cc:3037] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:45.703342 18618 raft_consensus.cc:2454] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4abd97fd5dd4c5cb98978b44ac9e932 in term 1.
I20260809 08:47:45.703466 18414 raft_consensus.cc:2454] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4abd97fd5dd4c5cb98978b44ac9e932 in term 1.
I20260809 08:47:45.703500 18103 leader_election.cc:302] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cd416f1399e4f8f9d3d0b9681f5b791, d4abd97fd5dd4c5cb98978b44ac9e932; no voters:
I20260809 08:47:45.703576 18721 raft_consensus.cc:2781] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:45.703742 18721 raft_consensus.cc:686] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [term 1 LEADER]: Becoming Leader. State: Replica: d4abd97fd5dd4c5cb98978b44ac9e932, State: Running, Role: LEADER
I20260809 08:47:45.703831 18721 consensus_queue.cc:237] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } }
I20260809 08:47:45.704576 17987 catalog_manager.cc:5495] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4abd97fd5dd4c5cb98978b44ac9e932 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4abd97fd5dd4c5cb98978b44ac9e932" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46241 } health_report { overall_health: HEALTHY } } }
I20260809 08:47:45.714008 18704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.713218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.10.184.65:46241)
0809 08:47:45.713257 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.713259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.713269 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:45.713326 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.713327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.713338 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.713381 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.713877 (+ 496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.713878 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.713878 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.713880 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.713890 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.713892 (+ 2us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.713926 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.713955 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.713964 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:45.715529 18618 raft_consensus.cc:1261] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 1 FOLLOWER]: Refusing update from remote peer d4abd97fd5dd4c5cb98978b44ac9e932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:45.715544 18414 raft_consensus.cc:1261] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 1 FOLLOWER]: Refusing update from remote peer d4abd97fd5dd4c5cb98978b44ac9e932: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:45.715835 18728 consensus_queue.cc:1035] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cd416f1399e4f8f9d3d0b9681f5b791" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:45.715955 18721 consensus_queue.cc:1035] T 5898ea0a15774ddd8c1cb2016f3d4685 P d4abd97fd5dd4c5cb98978b44ac9e932 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5431df10cbf74528b9349d9c6625f73b" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:45.718588 18733 mvcc.cc:205] Tried to move back new op lower bound from 7316542528368566272 to 7316542528323457024. Current Snapshot: MvccSnapshot[applied={T|T < 7316542528368566272}]
I20260809 08:47:45.719473 18734 mvcc.cc:205] Tried to move back new op lower bound from 7316542528368566272 to 7316542528323457024. Current Snapshot: MvccSnapshot[applied={T|T < 7316542528368566272}]
I20260809 08:47:45.719588 18736 mvcc.cc:205] Tried to move back new op lower bound from 7316542528368566272 to 7316542528323457024. Current Snapshot: MvccSnapshot[applied={T|T < 7316542528368566272 or (T in {7316542528368566272})}]
I20260809 08:47:45.720573 17987 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:45.720650 17987 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:45.721326 17987 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acbad63817644c8db7ddf6ac1d9eacb3: Sending DeleteTablet for 3 replicas of tablet 5898ea0a15774ddd8c1cb2016f3d4685
I20260809 08:47:45.721666 18587 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:45 UTC) from {username='slave'} at 127.0.0.1:52234
I20260809 08:47:45.721675 18174 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:45 UTC) from {username='slave'} at 127.0.0.1:37336
I20260809 08:47:45.721886 18386 tablet_service.cc:1514] Processing DeleteTablet for tablet 5898ea0a15774ddd8c1cb2016f3d4685 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:45 UTC) from {username='slave'} at 127.0.0.1:58448
I20260809 08:47:45.722110 18747 tablet_replica.cc:330] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: stopping tablet replica
I20260809 08:47:45.722230 18747 raft_consensus.cc:2229] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:45.722302 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 18085
I20260809 08:47:45.722332 18747 raft_consensus.cc:2258] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:45.739830 18747 ts_tablet_manager.cc:1905] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:45.740855 18745 tablet_replica.cc:330] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: stopping tablet replica
I20260809 08:47:45.740979 18745 raft_consensus.cc:2229] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:45.741096 18745 raft_consensus.cc:2258] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:45.741225 18747 ts_tablet_manager.cc:1918] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:45.741299 18747 log.cc:1196] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/5898ea0a15774ddd8c1cb2016f3d4685
I20260809 08:47:45.741524 18747 ts_tablet_manager.cc:1939] T 5898ea0a15774ddd8c1cb2016f3d4685 P 5431df10cbf74528b9349d9c6625f73b: Deleting consensus metadata
I20260809 08:47:45.741608 18745 ts_tablet_manager.cc:1905] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:45.741911 17972 catalog_manager.cc:4841] TS 5431df10cbf74528b9349d9c6625f73b (127.10.184.66:40799): tablet 5898ea0a15774ddd8c1cb2016f3d4685 (table test-table [id=fedc0ca30eaf45c4a06e53ab5b532550]) successfully deleted
I20260809 08:47:45.743054 18745 ts_tablet_manager.cc:1918] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:45.743131 18745 log.cc:1196] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/5898ea0a15774ddd8c1cb2016f3d4685
I20260809 08:47:45.743379 18745 ts_tablet_manager.cc:1939] T 5898ea0a15774ddd8c1cb2016f3d4685 P 4cd416f1399e4f8f9d3d0b9681f5b791: Deleting consensus metadata
I20260809 08:47:45.743763 17966 catalog_manager.cc:4841] TS 4cd416f1399e4f8f9d3d0b9681f5b791 (127.10.184.67:39139): tablet 5898ea0a15774ddd8c1cb2016f3d4685 (table test-table [id=fedc0ca30eaf45c4a06e53ab5b532550]) successfully deleted
W20260809 08:47:45.745760 17970 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:46241 (error 108)
I20260809 08:47:45.746244 18701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.746119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:46241 (local address 127.0.0.1:37344)
0809 08:47:45.746177 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.746218 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:46241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:47:45.746287 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 18284
W20260809 08:47:45.746371 17970 catalog_manager.cc:4584] TS d4abd97fd5dd4c5cb98978b44ac9e932 (127.10.184.65:46241): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5898ea0a15774ddd8c1cb2016f3d4685: Network error: Client connection negotiation failed: client connection to 127.10.184.65:46241: connect: Connection refused (error 111)
I20260809 08:47:45.752118 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 18481
I20260809 08:47:45.770020 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 17943
2026-08-09T08:47:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (989 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 08:47:45.794647 10977 mini_oidc.cc:152] Starting JWKS server
I20260809 08:47:45.795876 10977 webserver.cc:460] Webserver started at https://127.0.0.1:38227/ using document root <none> and password file <none>
I20260809 08:47:45.795962 10977 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:47:45.796121 10977 webserver.cc:460] Webserver started at http://127.0.0.1:43705/ using document root <none> and password file <none>
2026-08-09T08:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:45Z Disabled control of system clock
I20260809 08:47:45.804195 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40055
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:34129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:40055
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38227/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/testchainca.cert with env {}
W20260809 08:47:45.882496 18758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:45.882681 18758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:45.882700 18758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:45.883831 18758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:45.883870 18758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:45.883881 18758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:45.883893 18758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:45.883905 18758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:47:45.883929 18758 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:47:45.883951 18758 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38227/jwks.json
I20260809 08:47:45.885502 18758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:34129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:40055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38227/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:45.885722 18758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:45.885911 18758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:45.890450 18770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.890486 18766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:45.890486 18767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:45.890723 18758 server_base.cc:1034] running on GCE node
I20260809 08:47:45.890919 18758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:45.891136 18758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:45.892269 18758 hybrid_clock.cc:648] HybridClock initialized: now 1786265265892257 us; error 29 us; skew 500 ppm
I20260809 08:47:45.893537 18758 webserver.cc:460] Webserver started at http://127.10.184.126:41019/ using document root <none> and password file <none>
I20260809 08:47:45.893740 18758 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:45.893784 18758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:45.893899 18758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:45.894918 18758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "2bc577df8e174ae8ad4af7249eced602"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "8ff595dc704ebfd48146a7c3c52a2566"
server_key_iv: "e26fea06a9e25dd80a5b72377039af40"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.895291 18758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "2bc577df8e174ae8ad4af7249eced602"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "8ff595dc704ebfd48146a7c3c52a2566"
server_key_iv: "e26fea06a9e25dd80a5b72377039af40"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.896610 18758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:47:45.897302 18779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.897468 18758 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:45.897554 18758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "2bc577df8e174ae8ad4af7249eced602"
format_stamp: "Formatted at 2026-08-09 08:47:45 on dist-test-slave-ncp9"
server_key: "8ff595dc704ebfd48146a7c3c52a2566"
server_key_iv: "e26fea06a9e25dd80a5b72377039af40"
server_key_version: "encryptionkey@0"
I20260809 08:47:45.897607 18758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:45.924407 18758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:45.924665 18758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:45.924764 18758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:45.928424 18758 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:40055
I20260809 08:47:45.928462 18849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:40055 every 8 connection(s)
I20260809 08:47:45.928777 18758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:45.929848 18850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:45.932101 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602: Bootstrap starting.
I20260809 08:47:45.932670 18850 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:45.932929 18850 log.cc:824] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:45.933540 18850 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602: No bootstrap required, opened a new log
I20260809 08:47:45.935050 18850 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } }
I20260809 08:47:45.935194 18850 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:45.935221 18850 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bc577df8e174ae8ad4af7249eced602, State: Initialized, Role: FOLLOWER
I20260809 08:47:45.935326 18850 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } }
I20260809 08:47:45.935377 18850 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:45.935396 18850 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:45.935425 18850 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:45.935868 18850 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } }
I20260809 08:47:45.935953 18850 leader_election.cc:302] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2bc577df8e174ae8ad4af7249eced602; no voters:
I20260809 08:47:45.936084 18850 leader_election.cc:288] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:45.936136 18858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:45.936283 18858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [term 1 LEADER]: Becoming Leader. State: Replica: 2bc577df8e174ae8ad4af7249eced602, State: Running, Role: LEADER
I20260809 08:47:45.936326 18850 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:45.936386 18858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } }
I20260809 08:47:45.936734 18861 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2bc577df8e174ae8ad4af7249eced602. Latest consensus state: current_term: 1 leader_uuid: "2bc577df8e174ae8ad4af7249eced602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } } }
I20260809 08:47:45.936811 18859 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2bc577df8e174ae8ad4af7249eced602" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2bc577df8e174ae8ad4af7249eced602" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40055 } } }
I20260809 08:47:45.936822 18861 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:45.936883 18859 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:45.937064 18869 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:45.937556 18869 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:45.937976 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 18758
I20260809 08:47:45.938066 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:45.938256 10977 external_mini_cluster.cc:1428] Setting key a5dfbff65a6495feab6c8de9ef000f4c
I20260809 08:47:45.939193 18869 catalog_manager.cc:1347] Generated new cluster ID: 231c528d73544da0bee3024c07aa4f30
I20260809 08:47:45.939244 18869 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:45.944190 18889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.938929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.10.184.126:40055)
0809 08:47:45.939090 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.939099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.939129 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:47:45.939246 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.939254 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.939324 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.939461 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:45.939690 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.941064 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.942155 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.942784 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.942886 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:45.943532 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:45.943536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:45.943536 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:45.943736 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:45.943774 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:45.943787 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:45.943845 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:45.943931 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:47:45.944002 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":68,"threads_started":1}
I20260809 08:47:45.945127 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:45.948348 18869 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:45.948879 18869 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:45.951486 18869 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2bc577df8e174ae8ad4af7249eced602: Generated new TSK 0
I20260809 08:47:45.951628 18869 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:47:45.973639 18898 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 08:47:45.973778 18898 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 08:47:45.974004 18889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:45.958426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.10.184.126:40055)
0809 08:47:45.958468 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:45.958470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:45.958606 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:47:45.958705 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:45.958706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:45.958718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:45.958772 (+ 54us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:47:45.958902 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.959393 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.960135 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.960335 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.960388 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:45.960571 (+ 183us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:47:45.973784 (+ 13213us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:47:45.973963 (+ 179us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 08:47:45.974045 18889 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51594: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 08:47:45.974421 18896 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:45.958371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:40055 (local address 127.0.0.1:51594)
0809 08:47:45.958543 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:45.958553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:45.958612 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:45.958777 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:45.958777 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:45.958779 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:47:45.958847 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:45.958850 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.959479 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.959481 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:45.960060 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:45.960062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:45.960396 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:45.960397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:45.960460 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:45.960482 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:47:45.973937 (+ 13455us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 08:47:45.974050 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.184.126:40055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.10.184.126:40055: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38227/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 08:47:45.974807 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 18758
2026-08-09T08:47:46Z chronyd exiting
I20260809 08:47:46.021972 10977 test_util.cc:175] -----------------------------------------------
I20260809 08:47:46.022050 10977 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786265241374876-10977-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 08:47:46.022831 10977 mini_oidc.cc:152] Starting JWKS server
I20260809 08:47:46.023465 10977 webserver.cc:460] Webserver started at https://127.0.0.1:42369/ using document root <none> and password file <none>
I20260809 08:47:46.023505 10977 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:47:46.023747 10977 webserver.cc:460] Webserver started at http://127.0.0.1:35451/ using document root <none> and password file <none>
2026-08-09T08:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:46Z Disabled control of system clock
I20260809 08:47:46.032018 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40159
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:39471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:40159
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42369/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/testchainca.cert with env {}
W20260809 08:47:46.111163 18910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:46.111354 18910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:46.111383 18910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:46.112500 18910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:46.112538 18910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:46.112555 18910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:46.112569 18910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:46.112586 18910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:47:46.112608 18910 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:47:46.112624 18910 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42369/jwks.json
I20260809 08:47:46.114104 18910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:40159
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42369/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:46.114336 18910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:46.114526 18910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:46.119064 18918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.119056 18919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.119076 18923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:46.119365 18910 server_base.cc:1034] running on GCE node
I20260809 08:47:46.119585 18910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:46.119866 18910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:46.120990 18910 hybrid_clock.cc:648] HybridClock initialized: now 1786265266120971 us; error 33 us; skew 500 ppm
I20260809 08:47:46.122098 18910 webserver.cc:460] Webserver started at http://127.10.184.126:38121/ using document root <none> and password file <none>
I20260809 08:47:46.122305 18910 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:46.122349 18910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:46.122468 18910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:46.123538 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "afb13703daa84faeaaae84a57f108a87"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "d40bb621ea08dfdaa8c7e3dab85c3870"
server_key_iv: "49e9d045744c37e5d84905971d690695"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.123912 18910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "afb13703daa84faeaaae84a57f108a87"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "d40bb621ea08dfdaa8c7e3dab85c3870"
server_key_iv: "49e9d045744c37e5d84905971d690695"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.125303 18910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:46.125988 18939 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:46.126166 18910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:46.126245 18910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "afb13703daa84faeaaae84a57f108a87"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "d40bb621ea08dfdaa8c7e3dab85c3870"
server_key_iv: "49e9d045744c37e5d84905971d690695"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.126308 18910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:46.134495 18910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:46.134744 18910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:46.134857 18910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:46.138561 18910 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:40159
I20260809 08:47:46.138599 19019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:40159 every 8 connection(s)
I20260809 08:47:46.138890 18910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:46.139528 19021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:46.142350 19021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: Bootstrap starting.
I20260809 08:47:46.142876 19021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:46.143103 19021 log.cc:824] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:46.143704 19021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: No bootstrap required, opened a new log
I20260809 08:47:46.145287 19021 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } }
I20260809 08:47:46.145424 19021 raft_consensus.cc:374] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:46.145453 19021 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb13703daa84faeaaae84a57f108a87, State: Initialized, Role: FOLLOWER
I20260809 08:47:46.145619 19021 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } }
I20260809 08:47:46.145653 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 18910
I20260809 08:47:46.145702 19021 raft_consensus.cc:388] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:46.145741 19021 raft_consensus.cc:482] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:46.145756 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:46.145787 19021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:46.145936 10977 external_mini_cluster.cc:1428] Setting key fe219c0bc022f5f082edc9f09276125a
I20260809 08:47:46.146433 19021 raft_consensus.cc:504] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } }
I20260809 08:47:46.146550 19021 leader_election.cc:302] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb13703daa84faeaaae84a57f108a87; no voters:
I20260809 08:47:46.146711 19021 leader_election.cc:288] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:46.146744 19028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:46.146845 19028 raft_consensus.cc:686] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [term 1 LEADER]: Becoming Leader. State: Replica: afb13703daa84faeaaae84a57f108a87, State: Running, Role: LEADER
I20260809 08:47:46.146929 19028 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } }
I20260809 08:47:46.146972 19021 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:46.147265 19029 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afb13703daa84faeaaae84a57f108a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } } }
I20260809 08:47:46.147250 19028 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb13703daa84faeaaae84a57f108a87. Latest consensus state: current_term: 1 leader_uuid: "afb13703daa84faeaaae84a57f108a87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb13703daa84faeaaae84a57f108a87" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40159 } } }
I20260809 08:47:46.147439 19029 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:46.147459 19028 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87 [sys.catalog]: This master's current role is: LEADER
W20260809 08:47:46.148787 19050 catalog_manager.cc:1558] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:46.148881 19050 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:46.149046 19042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:46.149235 19042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:46.150523 19042 catalog_manager.cc:1347] Generated new cluster ID: e8c7efed920f4972918bcd1b1404b9fd
I20260809 08:47:46.150575 19042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:46.152089 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.146388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.10.184.126:40159)
0809 08:47:46.146615 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.146622 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.146650 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:47:46.146763 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.146770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.146839 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.147225 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:46.147322 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.148968 (+ 1646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.150049 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.150626 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.150759 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.151396 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:46.151399 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:46.151400 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:46.151600 (+ 200us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:46.151641 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:46.151656 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:46.151736 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:46.151792 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 08:47:46.151875 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":74,"threads_started":1}
I20260809 08:47:46.153096 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:46.171464 19042 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:46.172147 19042 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:46.177852 19042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afb13703daa84faeaaae84a57f108a87: Generated new TSK 0
I20260809 08:47:46.178012 19042 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 08:47:46.202844 19062 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 08:47:46.202913 19062 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 08:47:46.203258 19026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.189402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52570 (local address 127.10.184.126:40159)
0809 08:47:46.189454 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.189456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.189540 (+ 84us) server_negotiation.cc:407] Connection header received
0809 08:47:46.189592 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.189593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.189607 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.189684 (+ 77us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:47:46.189775 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.190518 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.191249 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.191563 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.191631 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.191771 (+ 140us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:47:46.203041 (+ 11270us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:47:46.203194 (+ 153us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 08:47:46.203301 19060 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:46.189331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:40159 (local address 127.0.0.1:52570)
0809 08:47:46.189481 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:46.189490 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:46.189543 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:46.189688 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:46.189688 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:46.189690 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:47:46.189738 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.189739 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.190611 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.190611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.191182 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.191183 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.191633 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.191633 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.191684 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.191688 (+ 4us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:47:46.203139 (+ 11451us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 08:47:46.203228 (+ 89us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.184.126:40159: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":89,"threads_started":1}
W20260809 08:47:46.203320 19026 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52570: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42369/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 08:47:46.203670 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 18910
2026-08-09T08:47:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (221 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 08:47:46.244650 10977 mini_oidc.cc:152] Starting JWKS server
I20260809 08:47:46.245456 10977 webserver.cc:460] Webserver started at https://127.0.0.1:39393/ using document root <none> and password file <none>
I20260809 08:47:46.245508 10977 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 08:47:46.245646 10977 webserver.cc:460] Webserver started at http://127.0.0.1:42773/ using document root <none> and password file <none>
2026-08-09T08:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:46Z Disabled control of system clock
I20260809 08:47:46.255120 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34939
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:37081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:34939
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39393/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:47:46.387107 19071 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:46.387326 19071 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:46.387347 19071 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:46.388490 19071 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:46.388531 19071 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:46.388543 19071 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:46.388566 19071 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:46.388589 19071 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:47:46.388612 19071 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:47:46.388633 19071 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39393/jwks.json
I20260809 08:47:46.390070 19071 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:37081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:34939
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39393/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19071
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:46.390307 19071 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:46.390588 19071 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:46.395268 19084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.395280 19082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.395292 19087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:46.395447 19071 server_base.cc:1034] running on GCE node
I20260809 08:47:46.395738 19071 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:46.396023 19071 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:46.397161 19071 hybrid_clock.cc:648] HybridClock initialized: now 1786265266397146 us; error 25 us; skew 500 ppm
I20260809 08:47:46.398284 19071 webserver.cc:460] Webserver started at http://127.10.184.126:40417/ using document root <none> and password file <none>
I20260809 08:47:46.398481 19071 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:46.398527 19071 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:46.398653 19071 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:46.399662 19071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "a96bfb12e52d41a286e69278846cda9f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "05281bf57d85e130890ec1a1dff5f77f"
server_key_iv: "9a1ac001bfeb5cdf3951fba185cf43be"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.400032 19071 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "a96bfb12e52d41a286e69278846cda9f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "05281bf57d85e130890ec1a1dff5f77f"
server_key_iv: "9a1ac001bfeb5cdf3951fba185cf43be"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.401427 19071 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:46.402034 19097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:46.402196 19071 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:46.402279 19071 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "a96bfb12e52d41a286e69278846cda9f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "05281bf57d85e130890ec1a1dff5f77f"
server_key_iv: "9a1ac001bfeb5cdf3951fba185cf43be"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.402338 19071 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:46.413947 19071 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:46.414189 19071 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:46.414305 19071 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:46.417968 19071 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:34939
I20260809 08:47:46.417999 19174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:34939 every 8 connection(s)
I20260809 08:47:46.418284 19071 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:46.419181 19175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:46.419937 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 19071
I20260809 08:47:46.420044 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:46.420208 10977 external_mini_cluster.cc:1428] Setting key 2f0231df57afcb1aa324eb8bf5dfdd55
I20260809 08:47:46.422330 19175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f: Bootstrap starting.
I20260809 08:47:46.423023 19175 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:46.423341 19175 log.cc:824] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:46.424024 19175 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f: No bootstrap required, opened a new log
I20260809 08:47:46.425462 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.420685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50490 (local address 127.10.184.126:34939)
0809 08:47:46.420855 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.420862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.420885 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:47:46.421350 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.421357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.421414 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.421520 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:46.421744 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.422639 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.423677 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.424122 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.424217 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.424856 (+ 639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:46.424859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:46.424859 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:46.425033 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:46.425064 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:46.425074 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:46.425121 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:46.425215 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 08:47:46.425278 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
I20260809 08:47:46.426402 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:46.426290 19175 raft_consensus.cc:348] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } }
I20260809 08:47:46.426446 19175 raft_consensus.cc:374] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:46.426473 19175 raft_consensus.cc:729] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a96bfb12e52d41a286e69278846cda9f, State: Initialized, Role: FOLLOWER
I20260809 08:47:46.426594 19175 consensus_queue.cc:260] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } }
I20260809 08:47:46.426676 19175 raft_consensus.cc:388] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:46.426713 19175 raft_consensus.cc:482] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:46.426770 19175 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:46.477636 19175 raft_consensus.cc:504] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } }
I20260809 08:47:46.477793 19175 leader_election.cc:302] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a96bfb12e52d41a286e69278846cda9f; no voters:
I20260809 08:47:46.478010 19175 leader_election.cc:288] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:46.478076 19187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:46.478259 19187 raft_consensus.cc:686] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [term 1 LEADER]: Becoming Leader. State: Replica: a96bfb12e52d41a286e69278846cda9f, State: Running, Role: LEADER
I20260809 08:47:46.478334 19175 sys_catalog.cc:564] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:46.478366 19187 consensus_queue.cc:237] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } }
I20260809 08:47:46.478793 19193 sys_catalog.cc:455] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader a96bfb12e52d41a286e69278846cda9f. Latest consensus state: current_term: 1 leader_uuid: "a96bfb12e52d41a286e69278846cda9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } } }
I20260809 08:47:46.478880 19193 sys_catalog.cc:458] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:46.479036 19188 sys_catalog.cc:455] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a96bfb12e52d41a286e69278846cda9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a96bfb12e52d41a286e69278846cda9f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34939 } } }
I20260809 08:47:46.479086 19188 sys_catalog.cc:458] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:46.479234 19199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:46.479748 19199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:46.481307 19199 catalog_manager.cc:1347] Generated new cluster ID: 241de1eff7af4369987334868fd74945
I20260809 08:47:46.481359 19199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:46.504725 19199 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:46.505466 19199 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:46.509565 19199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a96bfb12e52d41a286e69278846cda9f: Generated new TSK 0
I20260809 08:47:46.509728 19199 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:46.524489 19180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.521026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50492 (local address 127.10.184.126:34939)
0809 08:47:46.521084 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.521086 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.521204 (+ 118us) server_negotiation.cc:407] Connection header received
0809 08:47:46.521293 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.521294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.521303 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.521358 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:47:46.521507 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.521984 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.523148 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.523330 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.523373 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.523549 (+ 176us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:47:46.524286 (+ 737us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:47:46.524413 (+ 127us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50492: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:47:46.524545 19180 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50492: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 08:47:46.524554 19221 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:46.520970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34939 (local address 127.0.0.1:50492)
0809 08:47:46.521127 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:46.521139 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:46.521225 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:46.521375 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:46.521376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:46.521378 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:47:46.521451 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.521454 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.522078 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.522079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.523074 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.523078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.523394 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.523395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.523463 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.523469 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:47:46.524379 (+ 910us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 08:47:46.524464 (+ 85us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.184.126:34939: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39393/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":78,"threads_started":1}
I20260809 08:47:46.524973 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 19071
E20260809 08:47:46.543184 19223 webserver.cc:533] Webserver: sslize failed:
E20260809 08:47:46.543262 19223 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T08:47:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (325 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T08:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:46Z Disabled control of system clock
I20260809 08:47:46.578003 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:33531
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:45261
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:33531
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/testchainca.cert with env {}
W20260809 08:47:46.656353 19228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:46.656533 19228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:46.656554 19228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:46.657711 19228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:46.657752 19228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:46.657764 19228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:46.657778 19228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:46.657801 19228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 08:47:46.657824 19228 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 08:47:46.657845 19228 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 08:47:46.659307 19228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45261
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:33531
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:33531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19228
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:46.659521 19228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:46.659711 19228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:46.664369 19236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.664304 19243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:46.664405 19240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:46.664682 19228 server_base.cc:1034] running on GCE node
I20260809 08:47:46.664865 19228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:46.665154 19228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:46.666299 19228 hybrid_clock.cc:648] HybridClock initialized: now 1786265266666285 us; error 31 us; skew 500 ppm
I20260809 08:47:46.667455 19228 webserver.cc:460] Webserver started at http://127.10.184.126:40587/ using document root <none> and password file <none>
I20260809 08:47:46.667671 19228 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:46.667716 19228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:46.667833 19228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:46.668872 19228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "e17ff63d6fec48638983e73e658efd0f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "b0e79163424ccef184f04e575856e233"
server_key_iv: "4110d21d390d990270c027683834ca7b"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.669304 19228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "e17ff63d6fec48638983e73e658efd0f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "b0e79163424ccef184f04e575856e233"
server_key_iv: "4110d21d390d990270c027683834ca7b"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.670683 19228 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:46.671315 19256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:46.671450 19228 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:46.671541 19228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "e17ff63d6fec48638983e73e658efd0f"
format_stamp: "Formatted at 2026-08-09 08:47:46 on dist-test-slave-ncp9"
server_key: "b0e79163424ccef184f04e575856e233"
server_key_iv: "4110d21d390d990270c027683834ca7b"
server_key_version: "encryptionkey@0"
I20260809 08:47:46.671607 19228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:46.680248 19228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:46.680480 19228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:46.680590 19228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:46.684777 19228 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:33531
I20260809 08:47:46.684809 19338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:33531 every 8 connection(s)
I20260809 08:47:46.685158 19228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:46.686058 19340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:46.688159 19340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f: Bootstrap starting.
I20260809 08:47:46.688750 19340 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:46.689011 19340 log.cc:824] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:46.689656 19340 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f: No bootstrap required, opened a new log
I20260809 08:47:46.691222 19340 raft_consensus.cc:348] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } }
I20260809 08:47:46.691357 19340 raft_consensus.cc:374] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:46.691385 19340 raft_consensus.cc:729] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17ff63d6fec48638983e73e658efd0f, State: Initialized, Role: FOLLOWER
I20260809 08:47:46.691491 19340 consensus_queue.cc:260] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } }
I20260809 08:47:46.691584 19340 raft_consensus.cc:388] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:46.691609 19340 raft_consensus.cc:482] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:46.691654 19340 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:46.691670 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 19228
I20260809 08:47:46.691767 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:46.691929 10977 external_mini_cluster.cc:1428] Setting key 9acdbb496866e4dbaeda647d727cc819
I20260809 08:47:46.696254 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.692352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40216 (local address 127.10.184.126:33531)
0809 08:47:46.692534 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.692541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.692565 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:47:46.692658 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.692663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.692711 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.692809 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:46.692988 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.694078 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.694793 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.695166 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.695246 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.695721 (+ 475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:46.695724 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:46.695724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:47:46.695870 (+ 146us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:46.695901 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:47:46.695910 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:46.695958 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:46.696003 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:47:46.696071 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260809 08:47:46.697067 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:47:46.780053 19340 raft_consensus.cc:504] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } }
I20260809 08:47:46.780253 19340 leader_election.cc:302] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e17ff63d6fec48638983e73e658efd0f; no voters:
I20260809 08:47:46.780483 19340 leader_election.cc:288] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:46.780550 19350 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:46.780732 19350 raft_consensus.cc:686] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [term 1 LEADER]: Becoming Leader. State: Replica: e17ff63d6fec48638983e73e658efd0f, State: Running, Role: LEADER
I20260809 08:47:46.780776 19340 sys_catalog.cc:564] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:46.780843 19350 consensus_queue.cc:237] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } }
I20260809 08:47:46.781278 19353 sys_catalog.cc:455] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e17ff63d6fec48638983e73e658efd0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } } }
I20260809 08:47:46.781366 19353 sys_catalog.cc:458] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:46.781612 19354 sys_catalog.cc:455] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader e17ff63d6fec48638983e73e658efd0f. Latest consensus state: current_term: 1 leader_uuid: "e17ff63d6fec48638983e73e658efd0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17ff63d6fec48638983e73e658efd0f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 33531 } } }
I20260809 08:47:46.781930 19354 sys_catalog.cc:458] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:46.782140 19365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:46.782855 19365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:46.785265 19365 catalog_manager.cc:1347] Generated new cluster ID: f7e5870a17894359bc5c4ce8881d0878
I20260809 08:47:46.785316 19365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:46.814834 19365 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:46.815346 19365 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:46.820659 19365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e17ff63d6fec48638983e73e658efd0f: Generated new TSK 0
I20260809 08:47:46.820827 19365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:46.865274 19348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:46.827669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40220 (local address 127.10.184.126:33531)
0809 08:47:46.827716 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:46.827719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:46.827791 (+ 72us) server_negotiation.cc:407] Connection header received
0809 08:47:46.827859 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:46.827860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:46.827873 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:46.827933 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0809 08:47:46.828057 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.828524 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.829358 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.829543 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.829596 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.829758 (+ 162us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 08:47:46.864994 (+ 35236us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:47:46.865194 (+ 200us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40220: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 08:47:46.865348 19348 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40220: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 08:47:46.865352 19394 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:46.827593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:33531 (local address 127.0.0.1:40220)
0809 08:47:46.827727 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:46.827738 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:46.827800 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:46.827938 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:46.827939 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:46.827940 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0809 08:47:46.828001 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.828003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.828614 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.828615 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.829281 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:46.829285 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:46.829605 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:46.829606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:46.829669 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:46.829674 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 08:47:46.865142 (+ 35468us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 08:47:46.865270 (+ 128us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.10.184.126:33531: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260809 08:47:46.865722 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 19228
2026-08-09T08:47:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (321 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (90 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:48 dist-test-slave-ncp9 krb5kdc[19415](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:48 dist-test-slave-ncp9 krb5kdc[19415](info): set up 2 sockets
Aug 09 08:47:48 dist-test-slave-ncp9 krb5kdc[19415](info): commencing operation
krb5kdc: starting...
W20260809 08:47:49.183794 10977 mini_kdc.cc:121] Time spent starting KDC: real 2.113s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:49.292187 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44569
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44569
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:49.370411 19435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:49.370594 19435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:49.370615 19435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:49.371726 19435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:49.371764 19435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:49.371778 19435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:49.371788 19435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:49.371802 19435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:49.373209 19435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44569
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44569
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:49.373397 19435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:49.373557 19435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:49.378109 19441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.378098 19442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.378096 19445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:49.378122 19435 server_base.cc:1034] running on GCE node
I20260809 08:47:49.378594 19435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:49.378875 19435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:49.380014 19435 hybrid_clock.cc:648] HybridClock initialized: now 1786265269379999 us; error 27 us; skew 500 ppm
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.384884 19435 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:47:49.385300 19435 webserver.cc:460] Webserver started at http://127.10.184.126:44171/ using document root <none> and password file <none>
I20260809 08:47:49.385488 19435 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:49.385533 19435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:49.385653 19435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:49.386660 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "98f76393acd44621ad3f5e199330cdc1"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "9dc9562bb7bc0999a30610ea7735847a"
server_key_iv: "3cf591139ea9a7495681e4c398980305"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.387092 19435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "98f76393acd44621ad3f5e199330cdc1"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "9dc9562bb7bc0999a30610ea7735847a"
server_key_iv: "3cf591139ea9a7495681e4c398980305"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.388497 19435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:49.389189 19454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.389333 19435 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.389418 19435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "98f76393acd44621ad3f5e199330cdc1"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "9dc9562bb7bc0999a30610ea7735847a"
server_key_iv: "3cf591139ea9a7495681e4c398980305"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.389477 19435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:49.417023 19435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:49.419052 19435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:49.419188 19435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:49.422891 19435 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44569
I20260809 08:47:49.422924 19545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44569 every 8 connection(s)
I20260809 08:47:49.423236 19435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:49.424070 19550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:49.426286 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1: Bootstrap starting.
I20260809 08:47:49.426553 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 19435
I20260809 08:47:49.426659 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:49.426810 10977 external_mini_cluster.cc:1428] Setting key b7e37c019d9623b3892c3ac05d1fae50
I20260809 08:47:49.426920 19550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:49.427266 19550 log.cc:824] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:49.428045 19550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1: No bootstrap required, opened a new log
I20260809 08:47:49.430323 19550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } }
I20260809 08:47:49.430466 19550 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:49.430500 19550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98f76393acd44621ad3f5e199330cdc1, State: Initialized, Role: FOLLOWER
I20260809 08:47:49.430613 19550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } }
I20260809 08:47:49.430675 19550 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:49.430704 19550 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:49.430749 19550 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:49.431326 19550 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } }
I20260809 08:47:49.431432 19550 leader_election.cc:302] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98f76393acd44621ad3f5e199330cdc1; no voters:
I20260809 08:47:49.431576 19550 leader_election.cc:288] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:49.431615 19558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:49.431780 19558 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [term 1 LEADER]: Becoming Leader. State: Replica: 98f76393acd44621ad3f5e199330cdc1, State: Running, Role: LEADER
I20260809 08:47:49.431819 19550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:49.431947 19558 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } }
I20260809 08:47:49.432353 19561 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98f76393acd44621ad3f5e199330cdc1. Latest consensus state: current_term: 1 leader_uuid: "98f76393acd44621ad3f5e199330cdc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } } }
I20260809 08:47:49.432348 19560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98f76393acd44621ad3f5e199330cdc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98f76393acd44621ad3f5e199330cdc1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44569 } } }
I20260809 08:47:49.432498 19561 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:49.432504 19560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:49.432811 19575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:49.433279 19575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:49.435060 19575 catalog_manager.cc:1347] Generated new cluster ID: 0539e07b42bb4c54997a59278d5d7cf8
I20260809 08:47:49.435114 19575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:49.437494 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.427319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.10.184.126:44569)
0809 08:47:49.427485 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.427495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.427528 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:47:49.428049 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.428058 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.428118 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.428263 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.428828 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.430226 (+ 1398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.431061 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.431672 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.432064 (+ 392us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.434493 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.434496 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.434497 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.434508 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.436105 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.436420 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.436423 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.436423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.436509 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.436680 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.436681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.436681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.436928 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.437082 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.437146 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.437290 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":78,"threads_started":1}
I20260809 08:47:49.449314 19575 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:49.450152 19575 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:49.460402 19575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98f76393acd44621ad3f5e199330cdc1: Generated new TSK 0
I20260809 08:47:49.460546 19575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:47:49.489343 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:49.595440 19592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:49.595629 19592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:49.595661 19592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:49.596772 19592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:49.596812 19592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:49.596845 19592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:49.598270 19592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:49.598497 19592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:49.598690 19592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:49.603371 19605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.603376 19602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.603379 19600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:49.603717 19592 server_base.cc:1034] running on GCE node
I20260809 08:47:49.603907 19592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:49.604116 19592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:49.605247 19592 hybrid_clock.cc:648] HybridClock initialized: now 1786265269605232 us; error 27 us; skew 500 ppm
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.607599 19592 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:47:49.607983 19592 webserver.cc:460] Webserver started at http://127.10.184.65:44669/ using document root <none> and password file <none>
I20260809 08:47:49.608165 19592 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:49.608212 19592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:49.608335 19592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:49.609340 19592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "4f016d11d0a14c12b7f0f160232883af"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "e7a9d6564583541b3109f00afdfd8513"
server_key_iv: "f3d557ef4dd41e3d08a6c52112d699d0"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.609654 19592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "4f016d11d0a14c12b7f0f160232883af"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "e7a9d6564583541b3109f00afdfd8513"
server_key_iv: "f3d557ef4dd41e3d08a6c52112d699d0"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.610917 19592 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:49.611548 19612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.611704 19592 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:49.611788 19592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "4f016d11d0a14c12b7f0f160232883af"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "e7a9d6564583541b3109f00afdfd8513"
server_key_iv: "f3d557ef4dd41e3d08a6c52112d699d0"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.611843 19592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:49.621625 19592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:49.623234 19592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:49.623356 19592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:49.623544 19592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:49.623817 19592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:49.623852 19592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.623878 19592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:49.623894 19592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.629295 19592 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:42955
I20260809 08:47:49.629328 19790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:42955 every 8 connection(s)
I20260809 08:47:49.629588 19592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:49.634083 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 19592
I20260809 08:47:49.634186 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:49.634362 10977 external_mini_cluster.cc:1428] Setting key cd83fc7c6fa97e311b23da20d7d7af39
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:49.638772 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.630247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46243 (local address 127.10.184.126:44569)
0809 08:47:49.630301 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.630303 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.630473 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:47:49.631327 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.631329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.631342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.631386 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.632072 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.632641 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.633707 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.633904 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.633979 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.636210 (+ 2231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.636212 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.636212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.636219 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.637598 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.637953 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.637954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.637955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.638017 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.638215 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.638216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.638216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.638338 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.638401 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.638604 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.638686 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:47:49.638824 19794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.630211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44569 (local address 127.10.184.65:46243)
0809 08:47:49.630373 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:49.630405 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:49.631231 (+ 826us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:49.631448 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:49.631452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:49.631755 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:49.631991 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.631995 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.632728 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.632730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.633620 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.633622 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.634001 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.634003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.634125 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.634611 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:49.634621 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:49.636119 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:49.637674 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.637681 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.637682 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.637883 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.638073 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.638073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.638073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.638158 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.638393 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:49.638395 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:49.638496 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 08:47:49.638568 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:49.638680 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260809 08:47:49.639356 19791 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44569
I20260809 08:47:49.639439 19791 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:49.639595 19791 heartbeater.cc:511] Master 127.10.184.126:44569 requested a full tablet report, sending...
I20260809 08:47:49.640064 19477 ts_manager.cc:194] Registered new tserver with Master: 4f016d11d0a14c12b7f0f160232883af (127.10.184.65:42955)
I20260809 08:47:49.640633 19477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:46243
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:47:49.656858 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:49.735363 19798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:49.735613 19798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:49.735646 19798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:49.736766 19798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:49.736804 19798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:49.736833 19798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:49.738281 19798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:49.738483 19798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:49.738687 19798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:49.743465 19811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.743465 19805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.743498 19807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:49.743743 19798 server_base.cc:1034] running on GCE node
I20260809 08:47:49.743925 19798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:49.744119 19798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:49.745251 19798 hybrid_clock.cc:648] HybridClock initialized: now 1786265269745242 us; error 27 us; skew 500 ppm
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.747601 19798 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:47:49.748030 19798 webserver.cc:460] Webserver started at http://127.10.184.66:44499/ using document root <none> and password file <none>
I20260809 08:47:49.748229 19798 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:49.748270 19798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:49.748397 19798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:49.749401 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "b598b42cbb894e6d97486a9ca0979fde"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "807d6f7aa1139f7610c3b03ce5158658"
server_key_iv: "38c79816b806b5fce5b581a8dead5c60"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.749734 19798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "b598b42cbb894e6d97486a9ca0979fde"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "807d6f7aa1139f7610c3b03ce5158658"
server_key_iv: "38c79816b806b5fce5b581a8dead5c60"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.751183 19798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:49.751827 19819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.751971 19798 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:49.752050 19798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "b598b42cbb894e6d97486a9ca0979fde"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "807d6f7aa1139f7610c3b03ce5158658"
server_key_iv: "38c79816b806b5fce5b581a8dead5c60"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.752115 19798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:49.766629 19798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:49.768193 19798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:49.768321 19798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:49.768525 19798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:49.768805 19798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:49.768837 19798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.768857 19798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:49.768879 19798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.774382 19798 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43399
I20260809 08:47:49.774430 20002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43399 every 8 connection(s)
I20260809 08:47:49.774701 19798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:49.781085 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 19798
I20260809 08:47:49.781183 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:49.781371 10977 external_mini_cluster.cc:1428] Setting key aa5745508b39b55c3ae99a16cf3fac72
I20260809 08:47:49.783906 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.775442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:41915 (local address 127.10.184.126:44569)
0809 08:47:49.775507 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.775510 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.775665 (+ 155us) server_negotiation.cc:407] Connection header received
0809 08:47:49.776508 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.776509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.776519 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.776574 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.777167 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.777661 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.778551 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.778737 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.778785 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.781205 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.781207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.781207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.781216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.782735 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.783090 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.783092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.783092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.783156 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.783348 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.783349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.783350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.783471 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.783536 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.783737 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.783816 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:47:49.784020 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.775398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44569 (local address 127.10.184.66:41915)
0809 08:47:49.775547 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:49.775585 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:49.776422 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:49.776624 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:49.776626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:49.776862 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:49.777091 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.777096 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.777728 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.777729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.778483 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.778485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.778795 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.778795 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.778885 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.779342 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:49.779352 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:49.781086 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:49.782812 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.782818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.782820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.783025 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.783214 (+ 189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.783214 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.783215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.783300 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.783526 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:49.783528 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:49.783628 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 08:47:49.783703 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:47:49.783812 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260809 08:47:49.784708 20004 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44569
I20260809 08:47:49.784796 20004 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:49.784988 20004 heartbeater.cc:511] Master 127.10.184.126:44569 requested a full tablet report, sending...
I20260809 08:47:49.785348 19477 ts_manager.cc:194] Registered new tserver with Master: b598b42cbb894e6d97486a9ca0979fde (127.10.184.66:43399)
I20260809 08:47:49.785708 19477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:41915
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:47:49.804662 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:49.886113 20014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:49.886300 20014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:49.886332 20014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:49.887452 20014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:49.887491 20014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:49.887526 20014 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:49.889043 20014 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44569
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:49.889269 20014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:49.889468 20014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:49.894151 20025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.894225 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:49.894176 20027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:49.894773 20014 server_base.cc:1034] running on GCE node
I20260809 08:47:49.894963 20014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:49.895174 20014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:49.896317 20014 hybrid_clock.cc:648] HybridClock initialized: now 1786265269896304 us; error 29 us; skew 500 ppm
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.898702 20014 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:47:49.899117 20014 webserver.cc:460] Webserver started at http://127.10.184.67:39407/ using document root <none> and password file <none>
I20260809 08:47:49.899307 20014 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:49.899349 20014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:49.899463 20014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:49.900527 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "d4e2bbfdf5114c3eb625e8817a1535e2"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "17d1e96e0efe1565b0e7d41b3d1163ed"
server_key_iv: "767bf1623b1c91573ff749f577bb8747"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.900978 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "d4e2bbfdf5114c3eb625e8817a1535e2"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "17d1e96e0efe1565b0e7d41b3d1163ed"
server_key_iv: "767bf1623b1c91573ff749f577bb8747"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.902274 20014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:49.902978 20036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.903146 20014 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:49.903239 20014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "d4e2bbfdf5114c3eb625e8817a1535e2"
format_stamp: "Formatted at 2026-08-09 08:47:49 on dist-test-slave-ncp9"
server_key: "17d1e96e0efe1565b0e7d41b3d1163ed"
server_key_iv: "767bf1623b1c91573ff749f577bb8747"
server_key_version: "encryptionkey@0"
I20260809 08:47:49.903306 20014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:49.919541 20014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:49.921574 20014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:49.921708 20014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:49.921913 20014 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:49.922214 20014 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:49.922247 20014 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.922276 20014 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:49.922298 20014 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:49.927657 20014 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:39945
I20260809 08:47:49.927779 20222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:39945 every 8 connection(s)
I20260809 08:47:49.928017 20014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:49.934343 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 20014
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:47:49.934445 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:49.934635 10977 external_mini_cluster.cc:1428] Setting key 3dfbc34424d43f4f9acdfe31173b49c7
I20260809 08:47:49.937719 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.928746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:35583 (local address 127.10.184.126:44569)
0809 08:47:49.928808 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.928810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.928990 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:47:49.929886 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.929888 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.929902 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.929965 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.930519 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.931293 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.932377 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.932710 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.932785 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.934956 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.934958 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.934958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.934967 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.936458 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.936817 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.936818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.936818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.936881 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.937101 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.937101 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.937102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.937237 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.937301 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.937529 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.937615 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:47:49.937836 20228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.928696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44569 (local address 127.10.184.67:35583)
0809 08:47:49.928858 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:49.928919 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:49.929794 (+ 875us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:49.930033 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:49.930037 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:49.930266 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:49.930447 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.930451 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.931371 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.931372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.932280 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:49.932283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.932795 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.932797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:49.932945 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.933417 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:49.933426 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:49.934872 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:49.936532 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.936538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.936540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.936750 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.936949 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:49.936950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:49.936950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:49.937047 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:49.937291 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:49.937294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:49.937398 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 08:47:49.937483 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:47:49.937629 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":90,"threads_started":1}
I20260809 08:47:49.938211 20223 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44569
I20260809 08:47:49.938313 20223 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:49.938498 20223 heartbeater.cc:511] Master 127.10.184.126:44569 requested a full tablet report, sending...
I20260809 08:47:49.938843 19477 ts_manager.cc:194] Registered new tserver with Master: d4e2bbfdf5114c3eb625e8817a1535e2 (127.10.184.67:39945)
I20260809 08:47:49.939196 19477 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:35583
I20260809 08:47:49.946192 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/test-admin.keytab.
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.967011 10977 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 08:47:49.968650 20237 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:49.967873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40406 (local address 127.10.184.65:42955)
0809 08:47:49.968070 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.968076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.968090 (+ 14us) server_negotiation.cc:407] Connection header received
0809 08:47:49.968123 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.968125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.968144 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.968224 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.968564 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40406: BlockingRecv error: recv got EOF from 127.0.0.1:40406 (error 108)
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
W20260809 08:47:49.968654 10977 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.969826 10977 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.970777 10977 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:49.978008 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.971271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40414 (local address 127.10.184.65:42955)
0809 08:47:49.971329 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.971331 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.971353 (+ 22us) server_negotiation.cc:407] Connection header received
0809 08:47:49.971405 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.971406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.971416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.971460 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.971820 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.972538 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.973267 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.973581 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.973641 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.975671 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.975680 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.975681 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.975687 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.976958 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.977273 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.977277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.977277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.977334 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.977492 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.977493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.977493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.977620 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.977750 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.977828 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.977906 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 08:47:49.979501 20237 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:49.979097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40426 (local address 127.10.184.65:42955)
0809 08:47:49.979146 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.979148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.979156 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:49.979203 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.979204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.979213 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.979251 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.979456 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40426: BlockingRecv error: recv got EOF from 127.0.0.1:40426 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 08:47:49.979588 10977 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.980438 10977 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.981242 10977 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:49.987107 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.981683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.10.184.65:42955)
0809 08:47:49.981716 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.981718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.981762 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:47:49.981826 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.981827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.981836 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.981879 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.982218 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.982678 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.983317 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.983504 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.983565 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.985351 (+ 1786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.985353 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.985353 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.985359 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.986353 (+ 994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.986632 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.986632 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.986633 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.986678 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.986797 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.986798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.986798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.986870 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.986927 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.987003 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.987055 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:47:49.988379 20237 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:49.987960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40452 (local address 127.10.184.65:42955)
0809 08:47:49.988013 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.988015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.988038 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:47:49.988109 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.988110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.988117 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.988156 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.988339 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40452: BlockingRecv error: recv got EOF from 127.0.0.1:40452 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:47:49.988489 10977 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786265241374876-10977-0/krb5kdc/krb5cc' of type FILE
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.989358 10977 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:49.990095 10977 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 08:47:49 dist-test-slave-ncp9 krb5kdc[19415](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:47:49.995923 20237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:49.990503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40456 (local address 127.10.184.65:42955)
0809 08:47:49.990540 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:49.990542 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:49.990576 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:47:49.990639 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:49.990641 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:49.990648 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:49.990698 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:49.991013 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.991455 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.992240 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:49.992419 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:49.992469 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:49.994245 (+ 1776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:49.994247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:49.994247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:49.994253 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:49.995227 (+ 974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.995485 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.995486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.995486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.995525 (+ 39us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:49.995635 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:49.995636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:49.995636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:49.995705 (+ 69us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:49.995758 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:49.995822 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 08:47:49.995872 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:47:49.996374 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 19592
I20260809 08:47:50.013279 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 19798
I20260809 08:47:50.026646 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 20014
I20260809 08:47:50.034696 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 19435
2026-08-09T08:47:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2986 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:50 dist-test-slave-ncp9 krb5kdc[20254](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:50 dist-test-slave-ncp9 krb5kdc[20254](info): set up 2 sockets
Aug 09 08:47:50 dist-test-slave-ncp9 krb5kdc[20254](info): commencing operation
krb5kdc: starting...
W20260809 08:47:51.161898 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab.
Entry for principal oryx/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab.
I20260809 08:47:51.268589 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38185
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab
--principal=oryx/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:38185
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:51.347289 20274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:51.347472 20274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:51.347491 20274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:51.348575 20274 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:51.348615 20274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:51.348628 20274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:51.348658 20274 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:51.348682 20274 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:51.350123 20274 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:38185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab
--principal=oryx/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20274
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:51.350337 20274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:51.350528 20274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:51.355036 20279 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.355116 20280 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.355144 20282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.355155 20274 server_base.cc:1034] running on GCE node
I20260809 08:47:51.355489 20274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:51.355759 20274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:51.356914 20274 hybrid_clock.cc:648] HybridClock initialized: now 1786265271356882 us; error 26 us; skew 500 ppm
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:51.359107 20274 init.cc:377] Logged in from keytab as oryx/127.10.184.126@KRBTEST.COM (short username oryx)
I20260809 08:47:51.359506 20274 webserver.cc:460] Webserver started at http://127.10.184.126:36939/ using document root <none> and password file <none>
I20260809 08:47:51.359699 20274 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:51.359745 20274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:51.359864 20274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:51.360806 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "8a18dc7338874f0aadbbb3930f504831"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "b28d9be0b865a54752860ce2ce0f4c56"
server_key_iv: "2ac01b0e2e8c590705bcc2b7dccde9b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.361166 20274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "8a18dc7338874f0aadbbb3930f504831"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "b28d9be0b865a54752860ce2ce0f4c56"
server_key_iv: "2ac01b0e2e8c590705bcc2b7dccde9b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.362334 20274 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:51.362933 20296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.363080 20274 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:51.363161 20274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "8a18dc7338874f0aadbbb3930f504831"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "b28d9be0b865a54752860ce2ce0f4c56"
server_key_iv: "2ac01b0e2e8c590705bcc2b7dccde9b0"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.363226 20274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:51.387921 20274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:51.389496 20274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:51.389624 20274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:51.393450 20274 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:38185
I20260809 08:47:51.393481 20379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:38185 every 8 connection(s)
I20260809 08:47:51.393785 20274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:51.394701 20380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:51.396808 20380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Bootstrap starting.
I20260809 08:47:51.397362 20380 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:51.397588 20380 log.cc:824] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:51.398239 20380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: No bootstrap required, opened a new log
I20260809 08:47:51.399715 20380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } }
I20260809 08:47:51.399843 20380 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:51.399864 20380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a18dc7338874f0aadbbb3930f504831, State: Initialized, Role: FOLLOWER
I20260809 08:47:51.399966 20380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } }
I20260809 08:47:51.400012 20380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:51.400030 20380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:51.400059 20380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:51.400758 20380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } }
I20260809 08:47:51.400843 20380 leader_election.cc:302] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a18dc7338874f0aadbbb3930f504831; no voters:
I20260809 08:47:51.400988 20380 leader_election.cc:288] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:51.401029 20384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:51.401178 20384 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 LEADER]: Becoming Leader. State: Replica: 8a18dc7338874f0aadbbb3930f504831, State: Running, Role: LEADER
I20260809 08:47:51.401223 20380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:51.401264 20384 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } }
I20260809 08:47:51.401592 20386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a18dc7338874f0aadbbb3930f504831. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } }
I20260809 08:47:51.401577 20385 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } }
I20260809 08:47:51.401677 20386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:51.401703 20385 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:51.402110 20396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:51.402493 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 20274
I20260809 08:47:51.402591 20396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:51.402606 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:51.402812 10977 external_mini_cluster.cc:1428] Setting key 98a7b1ca924f8f6d78ac26c8e425667c
I20260809 08:47:51.404294 20396 catalog_manager.cc:1347] Generated new cluster ID: b19e39e8a5b34b578567f3562ade15e5
I20260809 08:47:51.404347 20396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:51.411926 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.403370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43356 (local address 127.10.184.126:38185)
0809 08:47:51.403526 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:51.403535 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:51.403566 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:47:51.403685 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:51.403692 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:51.403762 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:51.403890 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:51.404543 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.405555 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.406291 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.406708 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.406801 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.408875 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:51.408878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:51.408879 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:51.408922 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:51.410402 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.410714 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.410717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.410718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.410812 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.410982 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.410983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.410983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.411256 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:51.411389 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:51.411594 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:47:51.411741 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 08:47:51.419226 20396 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:51.419883 20396 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:51.423797 20396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Generated new TSK 0
I20260809 08:47:51.423942 20396 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
I20260809 08:47:51.455910 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:51.560366 20420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:51.560601 20420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:51.560623 20420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:51.562202 20420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:51.562258 20420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:51.562294 20420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:51.563792 20420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:51.564021 20420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:51.564236 20420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:51.570188 20425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.570230 20428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.570420 20426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.570612 20420 server_base.cc:1034] running on GCE node
I20260809 08:47:51.570785 20420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:51.570971 20420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:51.572135 20420 hybrid_clock.cc:648] HybridClock initialized: now 1786265271572124 us; error 28 us; skew 500 ppm
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:51.574424 20420 init.cc:377] Logged in from keytab as oryx/127.10.184.65@KRBTEST.COM (short username oryx)
I20260809 08:47:51.574854 20420 webserver.cc:460] Webserver started at http://127.10.184.65:33191/ using document root <none> and password file <none>
I20260809 08:47:51.575047 20420 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:51.575093 20420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:51.575217 20420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:51.576184 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "222e39e5c5ae0a5694d144e81a64907d"
server_key_iv: "8a99942899d7ebd29d5f55f3399c9fd4"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.576507 20420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "222e39e5c5ae0a5694d144e81a64907d"
server_key_iv: "8a99942899d7ebd29d5f55f3399c9fd4"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.577761 20420 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:47:51.578389 20444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.578531 20420 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:51.578615 20420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "222e39e5c5ae0a5694d144e81a64907d"
server_key_iv: "8a99942899d7ebd29d5f55f3399c9fd4"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.578679 20420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:51.609413 20420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:51.611256 20420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:51.611392 20420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:51.611605 20420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:51.611903 20420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:51.611936 20420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.611965 20420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:51.611984 20420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.617372 20420 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:45037
I20260809 08:47:51.617432 20627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:45037 every 8 connection(s)
I20260809 08:47:51.617722 20420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:51.621304 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 20420
I20260809 08:47:51.621412 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:51.621611 10977 external_mini_cluster.cc:1428] Setting key 080413cfef84207cbefb6ec2304eba57
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:51.627856 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.618472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:32809 (local address 127.10.184.126:38185)
0809 08:47:51.618532 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:51.618534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:51.618696 (+ 162us) server_negotiation.cc:407] Connection header received
0809 08:47:51.619150 (+ 454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:51.619151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:51.619164 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:51.619218 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:51.620071 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.620563 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.621820 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.622061 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.622111 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.624222 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:51.624224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:51.624224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:51.624232 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:51.626571 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.626960 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.626961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.626962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.627026 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.627242 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.627243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.627243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.627382 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:51.627456 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:51.627658 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 08:47:51.627746 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:47:51.627897 20630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.618413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:32809)
0809 08:47:51.618568 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:51.618610 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:51.619078 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:51.619270 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:51.619273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:51.619825 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:51.620007 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.620012 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.620624 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.620624 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.621740 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.621744 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.622134 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.622135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.622261 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.622711 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:51.622720 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:51.624132 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:51.626658 (+ 2526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.626667 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.626669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.626889 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.627087 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.627087 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.627088 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.627186 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.627440 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:51.627442 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:51.627545 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 08:47:51.627618 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:47:51.627733 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260809 08:47:51.628356 20628 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:51.628440 20628 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:51.628600 20628 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:51.629158 20327 ts_manager.cc:194] Registered new tserver with Master: c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:51.630016 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:32809
WARNING: no policy specified for oryx/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
I20260809 08:47:51.644879 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:51.724682 20634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:51.724885 20634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:51.724948 20634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:51.726063 20634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:51.726109 20634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:51.726146 20634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:51.727564 20634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:51.727787 20634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:51.727988 20634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:51.732821 20640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.733050 20634 server_base.cc:1034] running on GCE node
W20260809 08:47:51.732874 20642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.733024 20639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.733314 20634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:51.733534 20634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:51.734710 20634 hybrid_clock.cc:648] HybridClock initialized: now 1786265271734695 us; error 30 us; skew 500 ppm
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:51.737008 20634 init.cc:377] Logged in from keytab as oryx/127.10.184.66@KRBTEST.COM (short username oryx)
I20260809 08:47:51.737386 20634 webserver.cc:460] Webserver started at http://127.10.184.66:37687/ using document root <none> and password file <none>
I20260809 08:47:51.737571 20634 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:51.737617 20634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:51.737751 20634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:51.738705 20634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "9f1a7d050c40db12da721ef5821acea5"
server_key_iv: "0114b2ca7e61e08ceea01536b2254810"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.739028 20634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "9f1a7d050c40db12da721ef5821acea5"
server_key_iv: "0114b2ca7e61e08ceea01536b2254810"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.740191 20634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:51.740810 20657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.740978 20634 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:51.741060 20634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "9f1a7d050c40db12da721ef5821acea5"
server_key_iv: "0114b2ca7e61e08ceea01536b2254810"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.741127 20634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:51.763773 20634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:51.765498 20634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:51.765635 20634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:51.765847 20634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:51.766140 20634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:51.766177 20634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.766206 20634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:51.766232 20634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.771661 20634 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:41325
I20260809 08:47:51.771708 20838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:41325 every 8 connection(s)
I20260809 08:47:51.771999 20634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:51.778875 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 20634
I20260809 08:47:51.778970 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:51.779156 10977 external_mini_cluster.cc:1428] Setting key b530572f266af138f05834dfa830e48f
I20260809 08:47:51.780884 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.772689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:60723 (local address 127.10.184.126:38185)
0809 08:47:51.772773 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:51.772775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:51.772942 (+ 167us) server_negotiation.cc:407] Connection header received
0809 08:47:51.773463 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:51.773464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:51.773477 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:51.773529 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:51.774447 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.774948 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.775934 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.776135 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.776191 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.778262 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:51.778264 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:51.778264 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:51.778271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:51.779572 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.779994 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.779996 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.779996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.780059 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.780286 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.780287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.780287 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.780402 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:51.780458 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:51.780717 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 08:47:51.780798 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:47:51.781061 20841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.772621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:60723)
0809 08:47:51.772803 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:51.772848 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:51.773362 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:51.773597 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:51.773601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:51.774153 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:51.774347 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.774352 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.775032 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.775033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.775845 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.775847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.776207 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.776208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.776328 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.776791 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:51.776800 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:51.778179 (+ 1379us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:51.779641 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.779648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.779650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.779928 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.780113 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.780114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.780114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.780229 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.780457 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:51.780464 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:51.780596 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 08:47:51.780680 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:47:51.780832 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":86,"threads_started":1}
I20260809 08:47:51.781512 20839 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:51.781611 20839 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:51.781810 20839 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:51.782152 20327 ts_manager.cc:194] Registered new tserver with Master: f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
I20260809 08:47:51.782567 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:60723
WARNING: no policy specified for oryx/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
I20260809 08:47:51.803299 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:51.882910 20846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:51.883093 20846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:51.883124 20846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:51.884212 20846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:51.884251 20846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:51.884285 20846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:51.885787 20846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38185
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:51.886010 20846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:51.886217 20846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:51.890853 20864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.891012 20846 server_base.cc:1034] running on GCE node
W20260809 08:47:51.891081 20860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:51.891211 20862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:51.891373 20846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:51.891601 20846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:51.892735 20846 hybrid_clock.cc:648] HybridClock initialized: now 1786265271892724 us; error 30 us; skew 500 ppm
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:51.895022 20846 init.cc:377] Logged in from keytab as oryx/127.10.184.67@KRBTEST.COM (short username oryx)
I20260809 08:47:51.895401 20846 webserver.cc:460] Webserver started at http://127.10.184.67:42285/ using document root <none> and password file <none>
I20260809 08:47:51.895606 20846 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:51.895663 20846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:51.895788 20846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:51.896760 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "5787992ac5be4a1d8e69fe2a8d63e995"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "d5f8704bc803a139a2c568a6a752c9a4"
server_key_iv: "cf28da04ccf2fd6cdf930da56768a443"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.897111 20846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "5787992ac5be4a1d8e69fe2a8d63e995"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "d5f8704bc803a139a2c568a6a752c9a4"
server_key_iv: "cf28da04ccf2fd6cdf930da56768a443"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.898341 20846 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:51.899008 20874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.899169 20846 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:51.899255 20846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "5787992ac5be4a1d8e69fe2a8d63e995"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "d5f8704bc803a139a2c568a6a752c9a4"
server_key_iv: "cf28da04ccf2fd6cdf930da56768a443"
server_key_version: "encryptionkey@0"
I20260809 08:47:51.899307 20846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:51.907065 20846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:51.908631 20846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:51.908741 20846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:51.908932 20846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:51.909185 20846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:51.909214 20846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.909240 20846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:51.909255 20846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:51.914597 20846 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:39085
I20260809 08:47:51.914656 21044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:39085 every 8 connection(s)
I20260809 08:47:51.914911 20846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:51.917261 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 20846
I20260809 08:47:51.917346 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:51.917510 10977 external_mini_cluster.cc:1428] Setting key ffd25a61e2298b1388ef428c8d78e38e
Aug 09 08:47:51 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:51.923161 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.915493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:43905 (local address 127.10.184.126:38185)
0809 08:47:51.915564 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:51.915566 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:51.915740 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:47:51.916180 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:51.916181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:51.916192 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:51.916240 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:51.917110 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.917692 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.918848 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.919034 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.919083 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.921075 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:51.921076 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:51.921076 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:51.921083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:51.922141 (+ 1058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.922485 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.922486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.922486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.922529 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:51.922706 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:51.922707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:51.922707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:51.922797 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:51.922841 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:51.923033 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 08:47:51.923088 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:47:51.923301 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:51.915469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:43905)
0809 08:47:51.915618 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:51.915651 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:51.916105 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:51.916291 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:51.916294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:51.916821 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:51.917043 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.917048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.917794 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.917796 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.918781 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:51.918784 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:51.919090 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:51.919091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:51.919184 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:51.919681 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:51.919689 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:51.921000 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:51.922222 (+ 1222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.922228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.922229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.922429 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.922570 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:51.922571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:51.922571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:51.922656 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:51.922838 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:51.922841 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:51.922944 (+ 103us) client_negotiation.cc:769] Sending connection context
0809 08:47:51.923016 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:51.923126 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":81,"threads_started":1}
I20260809 08:47:51.923715 21046 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:51.923804 21046 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:51.923981 21046 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:51.924341 20327 ts_manager.cc:194] Registered new tserver with Master: 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67:39085)
I20260809 08:47:51.924669 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:43905
I20260809 08:47:51.929023 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.184.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.125.keytab.
Entry for principal oryx/127.10.184.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.125.keytab.
I20260809 08:47:51.951273 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:44471
--webserver_interface=127.10.184.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.125.keytab
--principal=oryx/127.10.184.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:38185,127.10.184.125:44471
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:52.033057 21053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:52.033257 21053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:52.033286 21053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:52.034391 21053 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:52.034443 21053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:52.034466 21053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:52.034484 21053 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:52.034503 21053 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:52.036001 21053 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:38185,127.10.184.125:44471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.125.keytab
--principal=oryx/127.10.184.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:44471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.184.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
W20260809 08:47:52.036119 21053 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.10.184.126:38185,127.10.184.125:44471'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 08:47:52.036270 21053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:52.036487 21053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.041143 21063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.041184 21064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.041152 21067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.041529 21053 server_base.cc:1034] running on GCE node
I20260809 08:47:52.041757 21053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.042052 21053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.043208 21053 hybrid_clock.cc:648] HybridClock initialized: now 1786265272043188 us; error 34 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.045550 21053 init.cc:377] Logged in from keytab as oryx/127.10.184.125@KRBTEST.COM (short username oryx)
I20260809 08:47:52.045925 21053 webserver.cc:460] Webserver started at http://127.10.184.125:36851/ using document root <none> and password file <none>
I20260809 08:47:52.046110 21053 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.046149 21053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.046247 21053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:52.047204 21053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/instance:
uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "394745bbdb3a5db895f03d759b9d57cf"
server_key_iv: "ff90f09bfd562cbff715ce4c5a4640e1"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.047494 21053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal/instance:
uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "394745bbdb3a5db895f03d759b9d57cf"
server_key_iv: "ff90f09bfd562cbff715ce4c5a4640e1"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.048763 21053 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:52.049460 21081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.049589 21053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.049670 21053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "394745bbdb3a5db895f03d759b9d57cf"
server_key_iv: "ff90f09bfd562cbff715ce4c5a4640e1"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.049734 21053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.067056 21053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.069034 21053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:52.069171 21053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.072671 21053 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.125:44471
I20260809 08:47:52.072708 21157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.125:44471 every 8 connection(s)
I20260809 08:47:52.073019 21053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
I20260809 08:47:52.073910 21160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:52.075245 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 21053
I20260809 08:47:52.075402 21160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.125@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
I20260809 08:47:52.084221 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.076082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43370 (local address 127.10.184.126:38185)
0809 08:47:52.076129 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.076131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.076327 (+ 196us) server_negotiation.cc:407] Connection header received
0809 08:47:52.076844 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.076845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.076858 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.076944 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.077430 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.077923 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.078939 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.079239 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.079298 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.081454 (+ 2156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.081456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.081456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.081464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.082818 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.083264 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.083265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.083265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.083330 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.083575 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.083576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.083576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.083714 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.083761 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.084050 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.084128 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:52.084300 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.076084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43370)
0809 08:47:52.076227 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.076261 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.076740 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.076964 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.076966 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.077200 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.077345 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.077349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.078011 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.078012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.078864 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.078867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.079303 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.079304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.079431 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.079855 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.079863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.081374 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.082890 (+ 1516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.082897 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.082899 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.083186 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.083389 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.083389 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.083390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.083517 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.083773 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.083778 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.083921 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.084017 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.084141 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":80,"threads_started":1}
I20260809 08:47:52.084857 21160 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:52.085304 21166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.076243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.10.184.125:44471)
0809 08:47:52.076349 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.076353 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.076381 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:47:52.076751 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.076760 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.076784 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.076866 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.077461 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.078464 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.079260 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.079675 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.079725 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.082376 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.082382 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.082382 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.082389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.083955 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.084340 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.084343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.084344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.084406 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.084610 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.084611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.084611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.084754 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.084882 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.085119 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.085207 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77,"thread_start_us":40,"threads_started":1}
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.125@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
I20260809 08:47:52.090843 21164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.085091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33574)
0809 08:47:52.085130 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.085139 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.085170 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.085352 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.085353 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.085486 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.085551 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.085553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.086191 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.086192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.086656 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.086657 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.086980 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.086981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.087039 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.087330 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.087334 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.088297 (+ 963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.090010 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.090011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.090011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.090156 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.090322 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.090323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.090323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.090375 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.090565 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.090566 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.090602 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.090662 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.090753 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":16}
I20260809 08:47:52.090952 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.085129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33574 (local address 127.10.184.125:44471)
0809 08:47:52.085259 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.085260 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.085265 (+ 5us) server_negotiation.cc:407] Connection header received
0809 08:47:52.085280 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.085281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.085289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.085340 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.085618 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.086122 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.086721 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.086928 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.086977 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.088366 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.088367 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.088367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.088373 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.089941 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.090223 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.090224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.090224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.090273 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.090423 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.090424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.090425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.090516 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.090569 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.090694 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.090829 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":114,"thread_start_us":93,"threads_started":1}
I20260809 08:47:52.092121 21160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Bootstrap starting.
I20260809 08:47:52.092715 21160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:52.093010 21160 log.cc:824] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:52.093729 21160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: No bootstrap required, opened a new log
I20260809 08:47:52.095036 21160 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } }
I20260809 08:47:52.095149 21160 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:52.095171 21160 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d9573e9b324791a2a16d9e38ea00c8, State: Initialized, Role: FOLLOWER
I20260809 08:47:52.095258 21160 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } }
I20260809 08:47:52.095527 21160 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:52.095485 21170 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } }
I20260809 08:47:52.095587 21170 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:52.097057 21189 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:52.097114 21189 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:52.098703 21053 master_runner.cc:428] Detected that this master e7d9573e9b324791a2a16d9e38ea00c8 is joining an existing cluster
I20260809 08:47:52.098776 21053 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.184.125:44471
I20260809 08:47:52.106837 21196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.099875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33588)
0809 08:47:52.099971 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.099981 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.100034 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.100177 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.100177 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.100339 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.100400 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.100402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.101312 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.101313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.102323 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.102326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.102768 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.102769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.102829 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.103221 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.103224 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.103480 (+ 256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.105294 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.105295 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.105296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.105767 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.106187 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.106188 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.106188 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.106257 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.106506 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.106507 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.106550 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.106628 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.106732 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":59,"threads_started":1}
I20260809 08:47:52.106917 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.099948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.10.184.125:44471)
0809 08:47:52.099981 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.099983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.100021 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:47:52.100091 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.100092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.100099 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.100123 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.100466 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.101240 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.102392 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.102686 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.102745 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.103554 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.103555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.103555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.103561 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.105221 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.106054 (+ 833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.106055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.106055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.106111 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.106320 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.106321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.106321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.106442 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.106508 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.106653 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.106810 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":64,"server-negotiator.queue_time_us":18}
I20260809 08:47:52.108029 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.099743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43374 (local address 127.10.184.126:38185)
0809 08:47:52.099787 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.099789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.099894 (+ 105us) server_negotiation.cc:407] Connection header received
0809 08:47:52.100053 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.100054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.100065 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.100116 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.100521 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.101274 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.103619 (+ 2345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.103862 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.103916 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.105158 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.105160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.105161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.105177 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.106522 (+ 1345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.107322 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.107323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.107323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.107388 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.107567 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.107568 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.107568 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.107675 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.107730 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.107875 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.107940 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:47:52.108043 21195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.099674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43374)
0809 08:47:52.099824 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.099836 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.099985 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.100127 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.100128 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.100350 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.100417 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.100420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.102415 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.102416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.103512 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.103515 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.103970 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.103971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.104032 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.104622 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.104627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.105058 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.106582 (+ 1524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.106583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.106583 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.106816 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.107463 (+ 647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.107464 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.107464 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.107512 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.107733 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.107734 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.107785 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.107845 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.107940 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.126@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
I20260809 08:47:52.116638 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.109905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33596)
0809 08:47:52.110008 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.110048 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.110113 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.110246 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.110248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.110473 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.110515 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.110516 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.111351 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.111352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.111896 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.111898 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.112330 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.112331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.112380 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.112770 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.112774 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.114247 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.115739 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.115746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.115747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.115929 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.116099 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.116099 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.116099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.116146 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.116393 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.116398 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.116435 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.116491 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.116573 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":58,"threads_started":1}
I20260809 08:47:52.116693 21168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.109964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.10.184.125:44471)
0809 08:47:52.110004 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.110006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.110106 (+ 100us) server_negotiation.cc:407] Connection header received
0809 08:47:52.110171 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.110172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.110181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.110243 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.110572 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.111289 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.111961 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.112269 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.112332 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.114326 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.114327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.114328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.114334 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.115668 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.115990 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.115991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.115991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.116043 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.116191 (+ 148us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.116191 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.116192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.116339 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.116405 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.116539 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.116612 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:47:52.117112 20327 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.184.125:44471: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
member_type: NON_VOTER
last_known_addr {
host: "127.10.184.125"
port: 44471
}
attrs {
promote: true
}
}
dest_uuid: "8a18dc7338874f0aadbbb3930f504831"
cas_config_opid_index: -1
I20260809 08:47:52.117488 20327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } }
I20260809 08:47:52.117949 21201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.118244 21119 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:52.118937 21201 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.184.125:44471
I20260809 08:47:52.118956 21119 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 08:47:52.119076 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.119143 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:52.119236 21053 master.cc:537] Master@127.10.184.125:44471 shutting down...
I20260809 08:47:52.119283 21201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:47:52.120990 20304 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 08:47:52.121418 20304 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 -> Peer e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125:44471): Couldn't send request to peer e7d9573e9b324791a2a16d9e38ea00c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:47:52.121882 21053 raft_consensus.cc:2229] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:52.121966 21053 raft_consensus.cc:2258] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:52.122013 21053 tablet_replica.cc:330] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: stopping tablet replica
I20260809 08:47:52.135353 21053 master.cc:552] Master@127.10.184.125:44471 shutdown complete.
I20260809 08:47:52.135398 21053 master_runner.cc:305] Clearing existing system tablet
I20260809 08:47:52.135520 21053 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.135557 21053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.136256 21053 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.002s sys 0.000s
I20260809 08:47:52.136718 21205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.136837 21053 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.136966 21053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.137001 21053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "394745bbdb3a5db895f03d759b9d57cf"
server_key_iv: "ff90f09bfd562cbff715ce4c5a4640e1"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.137842 21053 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:52.139237 21053 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:47:52.139307 21053 log.cc:1196] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 08:47:52.139497 21053 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Deleting consensus metadata
I20260809 08:47:52.139616 21053 master_runner.cc:315] Copying system tablet from 127.10.184.126:38185
I20260809 08:47:52.140249 21053 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: tablet copy: Beginning tablet copy session from remote peer at address 127.10.184.126:38185
I20260809 08:47:52.146386 21216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.140530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43378)
0809 08:47:52.140640 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.140650 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.140706 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.140836 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.140836 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.141013 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.141068 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.141070 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.142093 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.142094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.142638 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.142640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.143093 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.143094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.143147 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.143587 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.143590 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.143889 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.145544 (+ 1655us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.145545 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.145546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.145706 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.145888 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.145888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.145888 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.145932 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.146160 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.146161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.146201 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.146242 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.146314 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":61,"threads_started":1}
I20260809 08:47:52.146427 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.140608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.10.184.126:38185)
0809 08:47:52.140651 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.140653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.140696 (+ 43us) server_negotiation.cc:407] Connection header received
0809 08:47:52.140755 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.140755 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.140765 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.140832 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.141133 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.142011 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.142715 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.143021 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.143083 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.143978 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.143980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.143980 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.143988 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.145472 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.145770 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.145771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.145772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.145831 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.145985 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.145985 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.145986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.146107 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.146167 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.146267 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.146337 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:52.146711 20358 tablet_copy_service.cc:140] P 8a18dc7338874f0aadbbb3930f504831: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer e7d9573e9b324791a2a16d9e38ea00c8 ({username='oryx', principal='oryx/127.10.184.125@KRBTEST.COM'} at 127.0.0.1:43378)
I20260809 08:47:52.146844 20358 tablet_copy_service.cc:161] P 8a18dc7338874f0aadbbb3930f504831: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer e7d9573e9b324791a2a16d9e38ea00c8 at {username='oryx', principal='oryx/127.10.184.125@KRBTEST.COM'} at 127.0.0.1:43378: session id = e7d9573e9b324791a2a16d9e38ea00c8-00000000000000000000000000000000
I20260809 08:47:52.147498 20358 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:47:52.148052 21053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:47:52.149288 21053 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: tablet copy: Starting download of 0 data blocks...
I20260809 08:47:52.149432 21053 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: tablet copy: Starting download of 1 WAL segments...
I20260809 08:47:52.150279 21053 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:47:52.151111 20358 tablet_copy_service.cc:342] P 8a18dc7338874f0aadbbb3930f504831: Request end of tablet copy session e7d9573e9b324791a2a16d9e38ea00c8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.184.125@KRBTEST.COM'} at 127.0.0.1:43378
I20260809 08:47:52.151160 20358 tablet_copy_service.cc:433] P 8a18dc7338874f0aadbbb3930f504831: ending tablet copy session e7d9573e9b324791a2a16d9e38ea00c8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer e7d9573e9b324791a2a16d9e38ea00c8
I20260809 08:47:52.153673 21053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.154738 21218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.154932 21217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.154981 21053 server_base.cc:1034] running on GCE node
W20260809 08:47:52.155046 21220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.155187 21053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.155314 21053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.156451 21053 hybrid_clock.cc:648] HybridClock initialized: now 1786265272156440 us; error 22 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.157680 21053 init.cc:377] Logged in from keytab as oryx/127.10.184.125@KRBTEST.COM (short username oryx)
I20260809 08:47:52.157909 21053 webserver.cc:460] Webserver started at http://127.10.184.125:42059/ using document root <none> and password file <none>
I20260809 08:47:52.158012 21053 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.158044 21053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.158619 21053 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:52.159086 21229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.159219 21053 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.159263 21053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "394745bbdb3a5db895f03d759b9d57cf"
server_key_iv: "ff90f09bfd562cbff715ce4c5a4640e1"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.159317 21053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.172142 21053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.173588 21053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.176229 21053 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.125:44471
I20260809 08:47:52.176625 21326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.125:44471 every 8 connection(s)
I20260809 08:47:52.176662 21053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
I20260809 08:47:52.177620 21328 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:47:52.178188 21328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Bootstrap starting.
I20260809 08:47:52.181398 21328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:47:52.181669 21328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: Bootstrap complete.
I20260809 08:47:52.181879 21328 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } }
I20260809 08:47:52.181972 21328 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e7d9573e9b324791a2a16d9e38ea00c8, State: Initialized, Role: LEARNER
I20260809 08:47:52.182019 21328 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } }
I20260809 08:47:52.182230 21328 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:52.182226 21333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.182345 21333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:47:52.183751 21350 catalog_manager.cc:1259] Loaded cluster ID: b19e39e8a5b34b578567f3562ade15e5
I20260809 08:47:52.183794 21350 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: loading cluster ID for follower catalog manager: success
I20260809 08:47:52.184746 21350 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: acquiring CA information for follower catalog manager: success
I20260809 08:47:52.185062 21350 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:47:52.193336 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.186707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33616 (local address 127.10.184.125:44471)
0809 08:47:52.186792 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.186794 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.186804 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:52.186831 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.186831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.186839 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.186898 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.187588 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.188199 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.189260 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.189455 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.189497 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.190807 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.190808 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.190808 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.190814 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.192152 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.192562 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.192562 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.192563 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.192612 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.192790 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.192790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.192790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.192876 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.192993 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.193125 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.193224 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68,"thread_start_us":42,"threads_started":1}
WARNING: no policy specified for oryx/127.10.184.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.124.keytab.
Entry for principal oryx/127.10.184.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.124.keytab.
I20260809 08:47:52.216799 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:43729
--webserver_interface=127.10.184.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.124.keytab
--principal=oryx/127.10.184.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:52.296700 21357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:52.296880 21357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:52.296916 21357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:52.298063 21357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:52.298105 21357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:52.298120 21357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:52.298142 21357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:52.298164 21357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:52.299618 21357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.124.keytab
--principal=oryx/127.10.184.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:43729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.184.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:52.299824 21357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:52.300034 21357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.304860 21368 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.304905 21366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.304868 21365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.305076 21357 server_base.cc:1034] running on GCE node
I20260809 08:47:52.305274 21357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.305505 21357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.306663 21357 hybrid_clock.cc:648] HybridClock initialized: now 1786265272306649 us; error 28 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.308977 21357 init.cc:377] Logged in from keytab as oryx/127.10.184.124@KRBTEST.COM (short username oryx)
I20260809 08:47:52.309406 21357 webserver.cc:460] Webserver started at http://127.10.184.124:37317/ using document root <none> and password file <none>
I20260809 08:47:52.309619 21357 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.309664 21357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.309789 21357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:52.310782 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/instance:
uuid: "7ea2b5c9b35148d699ccb3d838887741"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "f381663bddbf5a41375d96744d9be98c"
server_key_iv: "6b8f3b4cbe16d6d953af8c5efe45829f"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.311105 21357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal/instance:
uuid: "7ea2b5c9b35148d699ccb3d838887741"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "f381663bddbf5a41375d96744d9be98c"
server_key_iv: "6b8f3b4cbe16d6d953af8c5efe45829f"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.312265 21357 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:47:52.312870 21378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.313022 21357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.313107 21357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "7ea2b5c9b35148d699ccb3d838887741"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "f381663bddbf5a41375d96744d9be98c"
server_key_iv: "6b8f3b4cbe16d6d953af8c5efe45829f"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.313156 21357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.342252 21357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.344221 21357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:52.344322 21357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.352254 21357 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.124:43729
I20260809 08:47:52.352288 21482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.124:43729 every 8 connection(s)
I20260809 08:47:52.352665 21357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
I20260809 08:47:52.353884 21483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:52.355916 21483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:47:52.361264 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 21357
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.124@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:52.364840 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.356686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.10.184.126:38185)
0809 08:47:52.356740 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.356742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.356925 (+ 183us) server_negotiation.cc:407] Connection header received
0809 08:47:52.357373 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.357374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.357385 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.357430 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.358214 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.358703 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.359764 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.359951 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.359994 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.362339 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.362341 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.362341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.362349 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.363590 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.364021 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.364022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.364022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.364083 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.364286 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.364286 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.364287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.364415 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.364479 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.364683 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.364763 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:47:52.365000 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.356514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43388)
0809 08:47:52.356803 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.356835 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.357292 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.357527 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.357530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.357831 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.358144 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.358149 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.358791 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.358793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.359688 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.359691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.360009 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.360010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.360107 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.360497 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.360506 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.362235 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.363673 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.363678 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.363680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.363955 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.364136 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.364137 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.364137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.364232 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.364470 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.364472 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.364571 (+ 99us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.364642 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.364762 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":180,"threads_started":1}
I20260809 08:47:52.365499 21483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.124@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
I20260809 08:47:52.370391 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.361813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.10.184.124:43729)
0809 08:47:52.361953 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.361957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.361977 (+ 20us) server_negotiation.cc:407] Connection header received
0809 08:47:52.361997 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.361999 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.362017 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.362310 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.362992 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.363840 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.364792 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.365301 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.365361 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.367568 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.367574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.367574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.367580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.369279 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.369582 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.369585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.369585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.369636 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.369860 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.369861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.369861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.369988 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.370120 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.370165 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.370237 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
I20260809 08:47:52.372063 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.365735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33620)
0809 08:47:52.365774 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.365782 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.365813 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.365976 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.365977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.366079 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.366137 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.366138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.367302 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.367303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.367859 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.367861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.368217 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.368218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.368328 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.368695 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.368700 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.369906 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.371396 (+ 1490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.371397 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.371397 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.371543 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.371685 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.371686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.371686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.371727 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.371877 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.371878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.371908 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.371949 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.372009 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 08:47:52.372082 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.365797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33620 (local address 127.10.184.125:44471)
0809 08:47:52.365860 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.365861 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.365869 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:52.365895 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.365895 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.365908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.365963 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.366211 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.366672 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.367942 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.368150 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.368211 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.369986 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.369988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.369988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.369996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.371334 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.371595 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.371596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.371596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.371639 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.371758 (+ 119us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.371759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.371759 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.371841 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.371880 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.371966 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.372024 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:47:52.372586 21483 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.124@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
I20260809 08:47:52.377707 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.372791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39398)
0809 08:47:52.372830 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.372839 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.372876 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.373004 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.373005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.373103 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.373129 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.373130 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.373720 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.373721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.374152 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.374153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.374447 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.374447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.374478 (+ 31us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.374798 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.374803 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.375765 (+ 962us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.376964 (+ 1199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.376965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.376965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.377112 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.377257 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.377257 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.377257 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.377339 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.377491 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.377491 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.377521 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.377563 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.377628 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:47:52.377766 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.372834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39398 (local address 127.10.184.124:43729)
0809 08:47:52.372865 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.372867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.372875 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:52.372955 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.372955 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.372964 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.373000 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.373176 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.373666 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.374205 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.374400 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.374442 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.375835 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.375836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.375836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.375843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.376862 (+ 1019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.377170 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.377171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.377171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.377214 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.377377 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.377377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.377377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.377453 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.377490 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.377579 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.377671 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":15}
I20260809 08:47:52.379276 21483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Bootstrap starting.
I20260809 08:47:52.380491 21483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:52.381251 21483 log.cc:824] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:52.381902 21483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: No bootstrap required, opened a new log
I20260809 08:47:52.384626 21483 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } }
I20260809 08:47:52.384749 21483 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:52.384770 21483 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea2b5c9b35148d699ccb3d838887741, State: Initialized, Role: FOLLOWER
I20260809 08:47:52.384860 21483 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } }
I20260809 08:47:52.385210 21483 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:52.385272 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } } }
I20260809 08:47:52.385368 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:47:52.387348 21504 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:52.387405 21504 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:52.389362 21357 master_runner.cc:428] Detected that this master 7ea2b5c9b35148d699ccb3d838887741 is joining an existing cluster
I20260809 08:47:52.389412 21357 master_runner.cc:432] Initiating AddMaster RPC to add 127.10.184.124:43729
I20260809 08:47:52.397425 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.390774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39414)
0809 08:47:52.391083 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.391093 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.391160 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.391308 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.391308 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.391547 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.391691 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.391694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.392364 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.392364 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.393080 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.393082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.393523 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.393524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.393615 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.394086 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.394091 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.394650 (+ 559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.396496 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.396497 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.396497 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.396622 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.396797 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.396797 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.396798 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.396851 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.397065 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.397066 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.397154 (+ 88us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.397218 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.397317 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":91,"threads_started":1}
I20260809 08:47:52.397465 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.390988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39414 (local address 127.10.184.124:43729)
0809 08:47:52.391031 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.391033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.391139 (+ 106us) server_negotiation.cc:407] Connection header received
0809 08:47:52.391218 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.391218 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.391225 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.391249 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.391774 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.392303 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.393150 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.393359 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.393415 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.395045 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.395046 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.395046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.395053 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.396432 (+ 1379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.396694 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.396695 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.396695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.396744 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.396917 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.396918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.396918 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.397013 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.397066 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.397247 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.397368 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":59,"server-negotiator.queue_time_us":29}
I20260809 08:47:52.403357 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.391207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33632 (local address 127.10.184.125:44471)
0809 08:47:52.391246 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.391248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.397891 (+ 6643us) server_negotiation.cc:407] Connection header received
0809 08:47:52.397959 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.397960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.397969 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.398022 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.398295 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.398755 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.399501 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.399707 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.399760 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.400657 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.400659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.400659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.400673 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.402078 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.402466 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.402467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.402468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.402552 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.402775 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.402776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.402776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.402913 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.403006 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.403227 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.403295 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:52.403393 21515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.391284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33632)
0809 08:47:52.397842 (+ 6558us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.397851 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.397913 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.398025 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.398025 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.398149 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.398225 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.398226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.398821 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.398822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.399426 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.399428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.399765 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.399766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.399861 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.400234 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.400238 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.400550 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.402143 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.402144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.402144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.402399 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.402606 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.402607 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.402607 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.402664 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.403072 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.403074 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.403121 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.403199 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.403299 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6548,"thread_start_us":147,"threads_started":1}
I20260809 08:47:52.415911 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.390712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.10.184.126:38185)
0809 08:47:52.390754 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.390756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.390999 (+ 243us) server_negotiation.cc:407] Connection header received
0809 08:47:52.391120 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.391121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.391131 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.391172 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.391983 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.392709 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.394028 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.394320 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.394363 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.413375 (+ 19012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.413376 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.413377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.413384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.414726 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.415148 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.415149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.415149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.415203 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.415413 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.415414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.415414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.415533 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.415601 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.415740 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.415820 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:52.415935 21514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.390687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43400)
0809 08:47:52.390930 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.390942 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.391052 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.391239 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.391239 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.391681 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.391863 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.391865 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.392807 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.392808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.393926 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.393929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.394439 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.394440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.394532 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.394969 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.394973 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.413290 (+ 18317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.414816 (+ 1526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.414817 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.414818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.415037 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.415264 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.415265 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.415265 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.415327 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.415606 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.415607 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.415649 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.415711 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.415822 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":348,"threads_started":1}
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265271, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.126@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
I20260809 08:47:52.421590 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 20420
I20260809 08:47:52.424398 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.417870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39424)
0809 08:47:52.417915 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.417938 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.417975 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.418195 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.418196 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.418336 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.418374 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.418375 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.418949 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.418950 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.419449 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.419450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.419787 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.419789 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.419852 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.420219 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.420225 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.421613 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.423358 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.423359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.423359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.423555 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.423759 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.423760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.423760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.423818 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.424104 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.424105 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.424150 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.424207 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.424303 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:47:52.424736 21486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.418032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.10.184.124:43729)
0809 08:47:52.418075 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.418076 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.418084 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:52.418114 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.418115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.418124 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.418168 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.418448 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.418869 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.419524 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.419717 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.419782 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.421709 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.421711 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.421711 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.421717 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.423262 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.423643 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.423644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.423644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.423696 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.423888 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.423889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.423889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.424015 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.424076 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.424578 (+ 502us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.424652 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 08:47:52.432468 20327 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.10.184.124:43729: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741"
member_type: NON_VOTER
last_known_addr {
host: "127.10.184.124"
port: 43729
}
attrs {
promote: true
}
}
dest_uuid: "8a18dc7338874f0aadbbb3930f504831"
cas_config_opid_index: 5
I20260809 08:47:52.432729 20327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
I20260809 08:47:52.433095 21198 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.433593 21426 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:52.434123 21198 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.10.184.124:43729
I20260809 08:47:52.434198 21198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.434266 21198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:52.434933 21426 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 08:47:52.435194 21201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:52.436380 21426 raft_consensus.cc:858] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
W20260809 08:47:52.436456 21426 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
I20260809 08:47:52.436942 21357 master.cc:537] Master@127.10.184.124:43729 shutting down...
I20260809 08:47:52.437340 21426 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.438530 21519 mvcc.cc:205] Tried to move back new op lower bound from 7316542551750791168 to 7316542551659929600. Current Snapshot: MvccSnapshot[applied={T|T < 7316542551668166656}]
I20260809 08:47:52.439229 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:45037
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:38185
--webserver_port=33191
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:47:52.439550 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a18dc7338874f0aadbbb3930f504831. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.439630 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:47:52.439677 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.439713 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:47:52.439736 21489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } }
I20260809 08:47:52.439764 21489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 08:47:52.440029 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.440107 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
W20260809 08:47:52.441653 20302 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 -> Peer 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124:43729): Couldn't send request to peer 7ea2b5c9b35148d699ccb3d838887741. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 08:47:52.441859 21357 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 08:47:52.441958 21357 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 08:47:52.441996 21357 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: stopping tablet replica
I20260809 08:47:52.443234 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.433209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33648)
0809 08:47:52.438296 (+ 5087us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.438309 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.438360 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.438574 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.438575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.438577 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:52.438615 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.438617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.440354 (+ 1737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.440355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.441380 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.441384 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.442980 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.442981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.443045 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.443070 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.443100 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.443138 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5069}
I20260809 08:47:52.443935 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.433311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33648 (local address 127.10.184.125:44471)
0809 08:47:52.435218 (+ 1907us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.435220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.438412 (+ 3192us) server_negotiation.cc:407] Connection header received
0809 08:47:52.438438 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.438438 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.438461 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.438516 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:52.438682 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.440202 (+ 1520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.442007 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.442878 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.443738 (+ 860us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.443817 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.443837 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.443875 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1881}
I20260809 08:47:52.444339 21278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 LEARNER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 08:47:52.444554 21198 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 08:47:52.445087 21278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.445953 21333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a18dc7338874f0aadbbb3930f504831. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.446031 21333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:47:52.446061 21333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: NON_VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: true } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.446105 21333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:47:52.446310 21198 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: attempting to promote NON_VOTER e7d9573e9b324791a2a16d9e38ea00c8 to VOTER
I20260809 08:47:52.446480 21198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
I20260809 08:47:52.446956 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.446845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39428)
0809 08:47:52.446887 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.446930 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:47:52.447000 21278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 LEARNER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 08:47:52.447157 21201 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 08:47:52.447292 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.447211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39430)
0809 08:47:52.447247 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.447270 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:43729: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 08:47:52.447346 20302 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 -> Peer 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124:43729): Couldn't send request to peer 7ea2b5c9b35148d699ccb3d838887741. Status: Network error: Client connection negotiation failed: client connection to 127.10.184.124:43729: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 08:47:52.447683 21201 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.447912 21278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.448582 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.448653 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:52.448705 21201 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.448745 21201 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:52.448956 21531 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.449052 21531 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:52.455577 21357 master.cc:552] Master@127.10.184.124:43729 shutdown complete.
I20260809 08:47:52.455616 21357 master_runner.cc:305] Clearing existing system tablet
I20260809 08:47:52.455737 21357 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.455777 21357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.456303 21357 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:52.456744 21536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.456871 21357 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.456940 21357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.456965 21357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "7ea2b5c9b35148d699ccb3d838887741"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "f381663bddbf5a41375d96744d9be98c"
server_key_iv: "6b8f3b4cbe16d6d953af8c5efe45829f"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.457538 21357 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:52.458568 21357 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 08:47:52.458629 21357 log.cc:1196] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 08:47:52.458853 21357 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Deleting consensus metadata
I20260809 08:47:52.458993 21357 master_runner.cc:315] Copying system tablet from 127.10.184.126:38185
I20260809 08:47:52.459528 21357 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: tablet copy: Beginning tablet copy session from remote peer at address 127.10.184.126:38185
I20260809 08:47:52.465004 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.459821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43404 (local address 127.10.184.126:38185)
0809 08:47:52.459859 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.459861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.459967 (+ 106us) server_negotiation.cc:407] Connection header received
0809 08:47:52.460043 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.460044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.460054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.460092 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.460396 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.460861 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.461547 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.461749 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.461800 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.462657 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.462659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.462659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.462666 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.463911 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.464246 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.464247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.464247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.464304 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.464466 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.464467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.464467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.464606 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.464665 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.464801 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.464879 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:52.465009 21547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.459780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43404)
0809 08:47:52.459916 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.459925 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.459979 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.460096 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.460097 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.460264 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.460303 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.460305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.460962 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.460963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.461477 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.461479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.461807 (+ 328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.461808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.461871 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.462291 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.462296 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.462577 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.463995 (+ 1418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.463996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.463996 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.464178 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.464356 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.464356 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.464357 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.464417 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.464657 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.464658 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.464712 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.464772 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.464873 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":101,"threads_started":1}
I20260809 08:47:52.465401 20358 tablet_copy_service.cc:140] P 8a18dc7338874f0aadbbb3930f504831: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7ea2b5c9b35148d699ccb3d838887741 ({username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:43404)
I20260809 08:47:52.465471 20358 tablet_copy_service.cc:161] P 8a18dc7338874f0aadbbb3930f504831: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7ea2b5c9b35148d699ccb3d838887741 at {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:43404: session id = 7ea2b5c9b35148d699ccb3d838887741-00000000000000000000000000000000
I20260809 08:47:52.465909 20358 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 08:47:52.466305 21357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 08:47:52.467141 21357 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: tablet copy: Starting download of 0 data blocks...
I20260809 08:47:52.467331 21357 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: tablet copy: Starting download of 1 WAL segments...
I20260809 08:47:52.469547 21357 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 08:47:52.470281 20358 tablet_copy_service.cc:342] P 8a18dc7338874f0aadbbb3930f504831: Request end of tablet copy session 7ea2b5c9b35148d699ccb3d838887741-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:43404
I20260809 08:47:52.470333 20358 tablet_copy_service.cc:433] P 8a18dc7338874f0aadbbb3930f504831: ending tablet copy session 7ea2b5c9b35148d699ccb3d838887741-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7ea2b5c9b35148d699ccb3d838887741
I20260809 08:47:52.472278 21357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.473330 21548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.473449 21357 server_base.cc:1034] running on GCE node
W20260809 08:47:52.473338 21551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.477006 21553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.477136 21357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.477262 21357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.478374 21357 hybrid_clock.cc:648] HybridClock initialized: now 1786265272478368 us; error 24 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.479489 21357 init.cc:377] Logged in from keytab as oryx/127.10.184.124@KRBTEST.COM (short username oryx)
I20260809 08:47:52.479714 21357 webserver.cc:460] Webserver started at http://127.10.184.124:45481/ using document root <none> and password file <none>
I20260809 08:47:52.479823 21357 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.479856 21357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.480394 21357 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.480865 21563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.481019 21357 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.481063 21357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "7ea2b5c9b35148d699ccb3d838887741"
format_stamp: "Formatted at 2026-08-09 08:47:52 on dist-test-slave-ncp9"
server_key: "f381663bddbf5a41375d96744d9be98c"
server_key_iv: "6b8f3b4cbe16d6d953af8c5efe45829f"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.481118 21357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.489961 21357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.491564 21357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.494553 21357 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.124:43729
I20260809 08:47:52.494825 21357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
I20260809 08:47:52.501001 21637 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.124:43729 every 8 connection(s)
I20260809 08:47:52.501796 21638 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:47:52.502334 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Bootstrap starting.
I20260809 08:47:52.504199 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:47:52.504412 21638 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Bootstrap complete.
I20260809 08:47:52.504590 21638 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
I20260809 08:47:52.504659 21638 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7ea2b5c9b35148d699ccb3d838887741, State: Initialized, Role: LEARNER
I20260809 08:47:52.504696 21638 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } }
I20260809 08:47:52.504908 21638 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:52.504866 21646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:52.505138 21646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: LEARNER
I20260809 08:47:52.506240 21660 catalog_manager.cc:1259] Loaded cluster ID: b19e39e8a5b34b578567f3562ade15e5
I20260809 08:47:52.506322 21660 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: loading cluster ID for follower catalog manager: success
I20260809 08:47:52.507355 21660 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: acquiring CA information for follower catalog manager: success
I20260809 08:47:52.507612 21660 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 08:47:52.559882 21529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:52.560128 21529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:52.560158 21529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:52.561870 21529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:52.561918 21529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:52.561952 21529 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:52.564148 21529 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:45037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=33191
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21529
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:52.564370 21529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:52.564602 21529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.570380 21674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.570487 21529 server_base.cc:1034] running on GCE node
W20260809 08:47:52.570534 21670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.570546 21671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.570770 21529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.570968 21529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.572118 21529 hybrid_clock.cc:648] HybridClock initialized: now 1786265272572109 us; error 26 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.574730 21529 init.cc:377] Logged in from keytab as oryx/127.10.184.65@KRBTEST.COM (short username oryx)
I20260809 08:47:52.575139 21529 webserver.cc:460] Webserver started at http://127.10.184.65:33191/ using document root <none> and password file <none>
I20260809 08:47:52.575348 21529 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.575407 21529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.576675 21529 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:52.577327 21685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.577486 21529 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:47:52.577564 21529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "222e39e5c5ae0a5694d144e81a64907d"
server_key_iv: "8a99942899d7ebd29d5f55f3399c9fd4"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.577863 21529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.601861 21529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.604259 21529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:52.604385 21529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.604707 21529 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:52.605079 21529 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:52.605120 21529 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.605140 21529 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:52.605162 21529 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.616175 21529 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:45037
I20260809 08:47:52.616202 21889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:45037 every 8 connection(s)
I20260809 08:47:52.616546 21529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:52.626368 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 21529
I20260809 08:47:52.626529 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 20634
I20260809 08:47:52.627754 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46013 (local address 127.10.184.125:44471)
0809 08:47:52.617837 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.617839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.618059 (+ 220us) server_negotiation.cc:407] Connection header received
0809 08:47:52.618509 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.618510 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.618522 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.618562 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.619481 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.620215 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.621852 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622138 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622212 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.625256 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.625258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.625259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.625267 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.626770 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.627090 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.627091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.627091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.627136 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.627293 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.627293 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.627294 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.627391 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.627422 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.627611 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.627680 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:47:52.627866 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:33391 (local address 127.10.184.124:43729)
0809 08:47:52.617457 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.617459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.617828 (+ 369us) server_negotiation.cc:407] Connection header received
0809 08:47:52.618478 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.618479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.618494 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.618639 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.620312 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.620983 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.621817 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622122 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622196 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.624585 (+ 2389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.624586 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.624586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.624592 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.626250 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.626706 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.626707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.626708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.626764 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.627000 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.627000 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.627001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.627160 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.627209 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.627663 (+ 454us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.627758 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":63,"threads_started":1}
I20260809 08:47:52.628183 21909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.10.184.65:46013)
0809 08:47:52.617850 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.617859 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.618382 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.619038 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.619039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.619234 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.619420 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.619422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.620372 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.620373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.621741 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.621744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622265 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622266 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.622357 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.622836 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.622846 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.625140 (+ 2294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.626847 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.626848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.626848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.627028 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.627200 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.627200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.627201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.627243 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.627452 (+ 209us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.627452 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.627485 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.627856 (+ 371us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.627998 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":16,"thread_start_us":60,"threads_started":1}
I20260809 08:47:52.628296 21900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.10.184.65:33391)
0809 08:47:52.617679 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.617722 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.618381 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.618686 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.618690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.619233 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.619349 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.619351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.621058 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.621059 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.621739 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.621741 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622213 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.622357 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.623002 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.623007 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.624492 (+ 1485us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.626338 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.626345 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.626348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.626620 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.626827 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.626827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.626828 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.626937 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.627277 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.627284 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.627478 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.627979 (+ 501us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.628102 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":75,"spinlock_wait_cycles":6400,"thread_start_us":185,"threads_started":1}
I20260809 08:47:52.628541 21893 heartbeater.cc:347] Connected to a master server at 127.10.184.125:44471
I20260809 08:47:52.628605 21893 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.628747 21893 heartbeater.cc:511] Master 127.10.184.125:44471 requested a full tablet report, sending...
I20260809 08:47:52.629235 21255 ts_manager.cc:194] Registered new tserver with Master: c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:52.639324 21890 heartbeater.cc:347] Connected to a master server at 127.10.184.124:43729
I20260809 08:47:52.639387 21890 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.639453 21890 heartbeater.cc:511] Master 127.10.184.124:43729 requested a full tablet report, sending...
I20260809 08:47:52.639876 21587 ts_manager.cc:194] Registered new tserver with Master: c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:52.640662 21902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:40653)
0809 08:47:52.617696 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.617725 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.618374 (+ 649us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.618768 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.618769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.619028 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.619345 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.619350 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.620965 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.620966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.621983 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.621986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622394 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.622466 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.622881 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.622884 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.625375 (+ 2491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.639662 (+ 14287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.639662 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.639663 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.639862 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.640120 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.640121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.640121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.640191 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.640431 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.640432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.640465 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.640509 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.640584 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":38,"thread_start_us":164,"threads_started":1}
I20260809 08:47:52.640712 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.617340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40653 (local address 127.10.184.126:38185)
0809 08:47:52.617391 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.617393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.617832 (+ 439us) server_negotiation.cc:407] Connection header received
0809 08:47:52.618694 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.618695 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.618706 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.618747 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.619440 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.620222 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622062 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.622317 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.622387 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.625819 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.625821 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.625821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.625829 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.639530 (+ 13701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.639953 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.639955 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.639955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.640024 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.640246 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.640246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.640247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.640377 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.640440 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.640543 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.640621 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:47:52.641072 21892 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:52.641130 21892 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.641191 21892 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:52.641472 20327 ts_manager.cc:194] Re-registered known tserver with Master: c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:52.641837 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:40653
I20260809 08:47:52.643242 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:41325
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:38185
--webserver_port=37687
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:52.725663 21910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:52.725864 21910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:52.725888 21910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:52.726966 21910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:52.727005 21910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:52.727032 21910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:52.728502 21910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:41325
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=37687
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:52.728686 21910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:52.728863 21910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.733850 21921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.733819 21918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.733832 21919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.733870 21910 server_base.cc:1034] running on GCE node
I20260809 08:47:52.734187 21910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.734390 21910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.735517 21910 hybrid_clock.cc:648] HybridClock initialized: now 1786265272735504 us; error 29 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.737907 21910 init.cc:377] Logged in from keytab as oryx/127.10.184.66@KRBTEST.COM (short username oryx)
I20260809 08:47:52.738292 21910 webserver.cc:460] Webserver started at http://127.10.184.66:37687/ using document root <none> and password file <none>
I20260809 08:47:52.738481 21910 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.738544 21910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.739782 21910 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:52.740280 21935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.740422 21910 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:52.740515 21910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "9f1a7d050c40db12da721ef5821acea5"
server_key_iv: "0114b2ca7e61e08ceea01536b2254810"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.740824 21910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.769209 21910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.770897 21910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:52.771030 21910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.771240 21910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:52.771536 21910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:52.771570 21910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.771598 21910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:52.771616 21910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.777662 21910 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:41325
I20260809 08:47:52.777793 22136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:41325 every 8 connection(s)
I20260809 08:47:52.778040 21910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:52.778270 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 21910
I20260809 08:47:52.778421 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 20846
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:52.789088 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:39085
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:38185
--webserver_port=42285
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
I20260809 08:47:52.792817 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.781031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:53753 (local address 127.10.184.124:43729)
0809 08:47:52.781192 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.781194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.781204 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:52.781974 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.781976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.781992 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.782038 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.783508 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.784216 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.785709 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786014 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786051 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.788814 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.788816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.788816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.788825 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.791221 (+ 2396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.791694 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.791696 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.791697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.791761 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792046 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.792047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.792048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.792197 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.792244 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.792604 (+ 360us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.792731 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260809 08:47:52.793460 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.781042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:41747 (local address 127.10.184.126:38185)
0809 08:47:52.781111 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.781113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.781299 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:47:52.781795 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.781796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.781824 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.781893 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.783179 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.784038 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.785697 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786000 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786053 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.789856 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.789857 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.789858 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.789866 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.791677 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792071 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.792073 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.792073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.792131 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792618 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.792620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.792620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.792755 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.792812 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.793297 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.793375 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:47:52.793458 22143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.778770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.10.184.66:53753)
0809 08:47:52.780869 (+ 2099us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.780932 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.781703 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.783170 (+ 1467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.783170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.783353 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.783407 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.783408 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.784298 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.784299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.785598 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.785601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786125 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786125 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.786251 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.786952 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.786958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.788610 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.791313 (+ 2703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.791319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.791322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.791603 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.791831 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.791832 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.791832 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.791963 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.792294 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.792302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.792459 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.793001 (+ 542us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.793220 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2054,"mutex_wait_us":163,"thread_start_us":2012,"threads_started":1}
I20260809 08:47:52.793458 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.778772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:41747)
0809 08:47:52.781225 (+ 2453us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.781235 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.781689 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.782217 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.782222 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.782847 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.783074 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.783079 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.784180 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.784182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.785598 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.785601 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786116 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.786251 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.786908 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.786919 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.789032 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.791766 (+ 2734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.791767 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.791768 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.791968 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.792388 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792390 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.792390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.792458 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.792838 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.792839 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.792880 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.792966 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.793146 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2409,"mutex_wait_us":80,"thread_start_us":2001,"threads_started":1}
I20260809 08:47:52.794178 22137 heartbeater.cc:347] Connected to a master server at 127.10.184.124:43729
I20260809 08:47:52.794178 22138 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:52.794283 22137 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.794284 22138 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.794463 22137 heartbeater.cc:511] Master 127.10.184.124:43729 requested a full tablet report, sending...
I20260809 08:47:52.794466 22138 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:52.794827 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.782243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:52783 (local address 127.10.184.125:44471)
0809 08:47:52.782292 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.782294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.784010 (+ 1716us) server_negotiation.cc:407] Connection header received
0809 08:47:52.784099 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.784100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.784111 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.784137 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.784522 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.785226 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786220 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786506 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786550 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.790679 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.790681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.790681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.790689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.792418 (+ 1729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792791 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.792792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.792793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.792856 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.794206 (+ 1350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.794207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.794207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.794344 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.794391 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.794663 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.794749 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:52.794816 20327 ts_manager.cc:194] Re-registered known tserver with Master: f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
I20260809 08:47:52.794929 21587 ts_manager.cc:194] Registered new tserver with Master: f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
I20260809 08:47:52.795285 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:41747
I20260809 08:47:52.795392 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.782064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.10.184.66:52783)
0809 08:47:52.783949 (+ 1885us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.783959 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.784034 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.784207 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.784208 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.784374 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.784423 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.784424 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.785319 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.785320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.786101 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.786104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.786632 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.786632 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.786694 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.787141 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.787146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.790562 (+ 3416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.792501 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.792502 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.792503 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.792719 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.794044 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.794045 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.794045 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.794112 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.794469 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.794470 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.794530 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.794994 (+ 464us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.795090 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1868,"thread_start_us":66,"threads_started":1}
I20260809 08:47:52.795854 22140 heartbeater.cc:347] Connected to a master server at 127.10.184.125:44471
I20260809 08:47:52.795904 22140 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.795958 22140 heartbeater.cc:511] Master 127.10.184.125:44471 requested a full tablet report, sending...
I20260809 08:47:52.796250 21255 ts_manager.cc:194] Registered new tserver with Master: f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
W20260809 08:47:52.872661 22148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:52.872846 22148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:52.872869 22148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:52.874023 22148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:52.874068 22148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:52.874096 22148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:52.875545 22148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:39085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=42285
--tserver_master_addrs=127.10.184.126:38185,127.10.184.125:44471,127.10.184.124:43729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:52.875756 22148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:52.875954 22148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:52.880581 22157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:52.880589 22158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.880959 22148 server_base.cc:1034] running on GCE node
W20260809 08:47:52.880599 22161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:52.881423 22148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:52.881616 22148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:52.882762 22148 hybrid_clock.cc:648] HybridClock initialized: now 1786265272882748 us; error 28 us; skew 500 ppm
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:52.885294 22148 init.cc:377] Logged in from keytab as oryx/127.10.184.67@KRBTEST.COM (short username oryx)
I20260809 08:47:52.885751 22148 webserver.cc:460] Webserver started at http://127.10.184.67:42285/ using document root <none> and password file <none>
I20260809 08:47:52.885944 22148 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:52.886051 22148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:52.887267 22148 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:52.887820 22172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.887979 22148 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.888074 22148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "5787992ac5be4a1d8e69fe2a8d63e995"
format_stamp: "Formatted at 2026-08-09 08:47:51 on dist-test-slave-ncp9"
server_key: "d5f8704bc803a139a2c568a6a752c9a4"
server_key_iv: "cf28da04ccf2fd6cdf930da56768a443"
server_key_version: "encryptionkey@0"
I20260809 08:47:52.888363 22148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:52.901889 22148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:52.903595 22148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:52.903728 22148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:52.903970 22148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:47:52.904263 22148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:52.904302 22148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.904330 22148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:52.904352 22148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:52.911170 22148 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:39085
I20260809 08:47:52.911248 22362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:39085 every 8 connection(s)
I20260809 08:47:52.911528 22148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:52.916761 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 22148
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.125@KRBTEST.COM
Aug 09 08:47:52 dist-test-slave-ncp9 krb5kdc[20254](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265272, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.124@KRBTEST.COM
I20260809 08:47:52.923760 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:52777)
0809 08:47:52.912823 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.912833 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.913859 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.914147 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.914148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.914505 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.914709 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.914716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915500 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.915500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.916664 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.916667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.917196 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.917197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.917298 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.917959 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.917969 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.920815 (+ 2846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.922505 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.922506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.922687 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.922873 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.922874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.922924 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.923191 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.923192 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.923259 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.923370 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.923538 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"mutex_wait_us":26,"thread_start_us":127,"threads_started":1}
I20260809 08:47:52.924060 20414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:52777 (local address 127.10.184.126:38185)
0809 08:47:52.912790 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.912793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.912885 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:47:52.914078 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.914079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.914088 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.914125 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.914818 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915324 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.916742 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.917051 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.917102 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.920923 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.920924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.920925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.920932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.922421 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922757 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.922758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.922758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.922811 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922986 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.922986 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.922987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.923104 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.923168 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.923892 (+ 724us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.923972 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:47:52.924222 21353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:53361 (local address 127.10.184.125:44471)
0809 08:47:52.912372 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.912374 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.912706 (+ 332us) server_negotiation.cc:407] Connection header received
0809 08:47:52.913957 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.913959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.913972 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.914024 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.914883 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915597 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.916486 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.916739 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.916803 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.920528 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.920530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.920530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.920539 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.922074 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922541 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.922543 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.922543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.922598 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922884 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.922885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.922885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.923009 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.923050 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.924067 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.924141 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:47:52.924449 22364 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38185
I20260809 08:47:52.924540 22364 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.924719 22364 heartbeater.cc:511] Master 127.10.184.126:38185 requested a full tablet report, sending...
I20260809 08:47:52.925107 20327 ts_manager.cc:194] Re-registered known tserver with Master: 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67:39085)
I20260809 08:47:52.925275 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.10.184.67:53361)
0809 08:47:52.912580 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.912614 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.913851 (+ 1237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.914083 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.914086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.914495 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.914806 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.914809 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915680 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.915681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.916403 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.916405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.916874 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.916875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.917169 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.917959 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.917969 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.920419 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.922164 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922171 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.922188 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.922462 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.922682 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.922683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.922683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.922808 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.923094 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.923101 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.923259 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.924320 (+ 1061us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.924417 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":31,"thread_start_us":173,"threads_started":1}
I20260809 08:47:52.925554 22367 heartbeater.cc:347] Connected to a master server at 127.10.184.125:44471
I20260809 08:47:52.925561 20327 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:52777
I20260809 08:47:52.925591 22367 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.925662 22367 heartbeater.cc:511] Master 127.10.184.125:44471 requested a full tablet report, sending...
I20260809 08:47:52.926033 21255 ts_manager.cc:194] Registered new tserver with Master: 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67:39085)
I20260809 08:47:52.926640 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:42191 (local address 127.10.184.124:43729)
0809 08:47:52.913036 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.913038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.913218 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:47:52.914171 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.914171 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.914180 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.914220 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.914796 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915563 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.916689 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.917003 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.917042 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.922519 (+ 5477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.922521 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.922521 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.922530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.924122 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.925075 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.925076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.925077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.925794 (+ 717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.926023 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.926024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.926024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.926149 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.926200 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.926466 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.926544 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:47:52.927295 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.912885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.10.184.67:42191)
0809 08:47:52.913150 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.913160 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.913857 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.914244 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.914245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.914495 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:52.914708 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.914713 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.915642 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.915643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.916575 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.916578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.917150 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.917151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.917291 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.918212 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:52.918217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:52.921535 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:52.924704 (+ 3169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.924704 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.924705 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.924986 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.925888 (+ 902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:52.925888 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:52.925889 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:52.925952 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:52.926283 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:52.926284 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:52.926328 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.927028 (+ 700us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.927145 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":5,"thread_start_us":211,"threads_started":1}
I20260809 08:47:52.927951 22363 heartbeater.cc:347] Connected to a master server at 127.10.184.124:43729
I20260809 08:47:52.928004 22363 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:52.928067 22363 heartbeater.cc:511] Master 127.10.184.124:43729 requested a full tablet report, sending...
I20260809 08:47:52.928642 21587 ts_manager.cc:194] Registered new tserver with Master: 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67:39085)
I20260809 08:47:52.930076 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.918794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33652 (local address 127.10.184.125:44471)
0809 08:47:52.919094 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.919096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.920475 (+ 1379us) server_negotiation.cc:407] Connection header received
0809 08:47:52.920498 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.920499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.920511 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.920562 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.921555 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.922327 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.924292 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.924759 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.924800 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.927096 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.927097 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.927098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.927105 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.928818 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.929216 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.929217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.929217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.929284 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.929480 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.929481 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.929481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.929606 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.929704 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.929815 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.929984 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":57,"threads_started":1}
I20260809 08:47:52.930500 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.917828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.10.184.124:43729)
0809 08:47:52.918065 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.918066 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.918245 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:47:52.918267 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.918268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.918279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.918336 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.919248 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.919979 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.924803 (+ 4824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.925096 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.925162 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.927190 (+ 2028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.927191 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.927191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.927198 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.929087 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.929442 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.929443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.929443 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.929495 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.929693 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.929694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.929695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.929829 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.929912 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.930070 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.930417 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":62,"threads_started":1}
I20260809 08:47:52.931555 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.918080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.10.184.126:38185)
0809 08:47:52.919397 (+ 1317us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.919399 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.919407 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:52.919459 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.919459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.919471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.919563 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:52.920321 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.922406 (+ 2085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.925749 (+ 3343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.926460 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.926523 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.928833 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:52.928834 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:52.928835 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:52.928841 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:52.930626 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.931004 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.931006 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.931006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.931065 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:52.931199 (+ 134us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:52.931200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:52.931200 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:52.931291 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:52.931352 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.931419 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.931480 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1216,"thread_start_us":152,"threads_started":1}
I20260809 08:47:52.932806 20327 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:52.934317 20327 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:52.942523 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.10.184.67:39085)
0809 08:47:52.938821 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.938826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.938837 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:47:52.938871 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.938874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.938887 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.938954 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:52.939095 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940435 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.941495 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.942066 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.942147 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.942361 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.942393 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.942445 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":117,"threads_started":1}
I20260809 08:47:52.942677 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39085 (local address 127.0.0.1:43206)
0809 08:47:52.938643 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.938653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.938699 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.938975 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.938976 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.938978 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:52.939016 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.939017 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940543 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.940544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.941399 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.941402 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.942152 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.942153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.942228 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.942232 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.942273 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.942311 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"mutex_wait_us":17,"thread_start_us":182,"threads_started":1}
I20260809 08:47:52.943172 21197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41325 (local address 127.0.0.1:57948)
0809 08:47:52.938156 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.938166 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.938209 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.939178 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.939179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.939180 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:52.939217 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.939219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940726 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.940727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.941844 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.941847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.942922 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.942923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.942995 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.942999 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.943064 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.943105 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":4}
I20260809 08:47:52.943384 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57948 (local address 127.10.184.66:41325)
0809 08:47:52.939048 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.939052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.939059 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:52.939088 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.939089 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.939099 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.939161 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:52.939306 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940648 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.941942 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.942707 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.942776 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.942891 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.943103 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.943149 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":149,"threads_started":1}
I20260809 08:47:52.943423 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35908 (local address 127.10.184.65:45037)
0809 08:47:52.938837 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:52.938840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:52.939306 (+ 466us) server_negotiation.cc:407] Connection header received
0809 08:47:52.939409 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:52.939411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:52.939425 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:52.939615 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:52.939714 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940601 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.941941 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.942843 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.942908 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.942997 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:52.943215 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 08:47:52.943344 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":179,"threads_started":1}
I20260809 08:47:52.943523 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:52.938395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:45037 (local address 127.0.0.1:35908)
0809 08:47:52.939248 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:52.939257 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:52.939306 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:52.939620 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:52.939621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:52.939622 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:52.939660 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.939661 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.940669 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.940669 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.941845 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:52.941847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:52.943033 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:52.943034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:52.943088 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:52.943091 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:52.943141 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:47:52.943173 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":837,"mutex_wait_us":26,"thread_start_us":198,"threads_started":1}
I20260809 08:47:52.943444 22245 tablet_service.cc:1467] Processing CreateTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 (DEFAULT_TABLE table=test-table [id=f083bd835c54497997e4cc93291c297e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:52.943842 22245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aacba1da5c1643d5aa7ddfa760c811d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:52.944295 21774 tablet_service.cc:1467] Processing CreateTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 (DEFAULT_TABLE table=test-table [id=f083bd835c54497997e4cc93291c297e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:52.944267 22011 tablet_service.cc:1467] Processing CreateTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 (DEFAULT_TABLE table=test-table [id=f083bd835c54497997e4cc93291c297e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:52.944634 21774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aacba1da5c1643d5aa7ddfa760c811d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:52.944676 22011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aacba1da5c1643d5aa7ddfa760c811d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:52.948509 22406 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Bootstrap starting.
I20260809 08:47:52.949154 22409 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Bootstrap starting.
I20260809 08:47:52.949322 22406 tablet_bootstrap.cc:654] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:52.949640 22406 log.cc:824] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:52.949990 22409 tablet_bootstrap.cc:654] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:52.950237 22408 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Bootstrap starting.
I20260809 08:47:52.950446 22406 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: No bootstrap required, opened a new log
I20260809 08:47:52.950523 22406 ts_tablet_manager.cc:1397] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:52.950528 22409 log.cc:824] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:52.951314 22409 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: No bootstrap required, opened a new log
I20260809 08:47:52.951355 22408 tablet_bootstrap.cc:654] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:52.951385 22409 ts_tablet_manager.cc:1397] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:52.951838 22408 log.cc:824] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:52.952585 22408 tablet_bootstrap.cc:492] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: No bootstrap required, opened a new log
I20260809 08:47:52.952648 22408 ts_tablet_manager.cc:1397] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:52.954344 22406 raft_consensus.cc:348] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.954397 22409 raft_consensus.cc:348] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.954521 22409 raft_consensus.cc:374] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:52.954514 22406 raft_consensus.cc:374] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:52.954551 22409 raft_consensus.cc:729] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23d117eca2d4ddc8f8c786e5b39b616, State: Initialized, Role: FOLLOWER
I20260809 08:47:52.954551 22406 raft_consensus.cc:729] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4dbbdabe71a486b8f7f1ab425f204bd, State: Initialized, Role: FOLLOWER
I20260809 08:47:52.954684 22406 consensus_queue.cc:260] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.954684 22409 consensus_queue.cc:260] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.954937 22409 ts_tablet_manager.cc:1428] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 08:47:52.954998 22406 ts_tablet_manager.cc:1428] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260809 08:47:52.955127 21892 heartbeater.cc:503] Master 127.10.184.126:38185 was elected leader, sending a full tablet report...
I20260809 08:47:52.955077 22408 raft_consensus.cc:348] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.955235 22408 raft_consensus.cc:374] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:52.955276 22408 raft_consensus.cc:729] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5787992ac5be4a1d8e69fe2a8d63e995, State: Initialized, Role: FOLLOWER
I20260809 08:47:52.955389 22408 consensus_queue.cc:260] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:52.955638 22408 ts_tablet_manager.cc:1428] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:47:52.955984 22364 heartbeater.cc:503] Master 127.10.184.126:38185 was elected leader, sending a full tablet report...
I20260809 08:47:52.956638 22138 heartbeater.cc:503] Master 127.10.184.126:38185 was elected leader, sending a full tablet report...
I20260809 08:47:53.024529 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.022148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.10.184.124:43729)
0809 08:47:53.022215 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.022217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.022225 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:53.022254 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.022255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.022277 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.022333 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:53.022457 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.022938 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.023813 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.024220 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.024285 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.024371 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.024413 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.024453 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:47:53.024531 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.022070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39446)
0809 08:47:53.022123 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:53.022137 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:53.022199 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:53.022341 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:53.022343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:53.022344 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:53.022397 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.022399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.023009 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.023011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.023726 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.023728 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.024295 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.024296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.024356 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.024359 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:53.024408 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:47:53.024452 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260809 08:47:53.024868 21598 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260809 08:47:53.025272 22419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a18dc7338874f0aadbbb3930f504831. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:53.025393 22419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: LEARNER
W20260809 08:47:53.032444 22141 tablet.cc:2369] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:53.055135 22417 raft_consensus.cc:482] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:53.055251 22417 raft_consensus.cc:504] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:53.055706 22417 leader_election.cc:288] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325), c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:53.059348 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.056036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:50281 (local address 127.10.184.66:41325)
0809 08:47:53.056099 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.056101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.056109 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:53.056133 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.056134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.056141 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.056201 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:53.056321 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.057034 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.058399 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.058954 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.059020 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.059029 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.059240 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.059280 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:47:53.059348 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.055948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:55361 (local address 127.10.184.65:45037)
0809 08:47:53.056008 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.056010 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.056019 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:53.056054 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.056054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.056064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.056121 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:53.056274 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.057055 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.058399 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.059046 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.059096 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.059103 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.059246 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.059280 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:47:53.059360 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.055883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:45037 (local address 127.10.184.67:55361)
0809 08:47:53.055956 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:53.055967 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:53.056010 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:53.056128 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:53.056129 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:53.056130 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:53.056210 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.056212 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.057111 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.057112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.058322 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.058325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.059108 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.059109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.059186 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.059189 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:53.059237 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:47:53.059269 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":56}
I20260809 08:47:53.059434 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.055873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41325 (local address 127.10.184.67:50281)
0809 08:47:53.055941 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:53.055952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:53.055991 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:53.056218 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:53.056218 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:53.056220 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:53.056262 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.056263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.057108 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.057109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.058322 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:53.058325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.059027 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.059028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:53.059163 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.059168 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:53.059225 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:47:53.059270 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":7}
I20260809 08:47:53.059762 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aacba1da5c1643d5aa7ddfa760c811d8" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" is_pre_election: true
I20260809 08:47:53.059845 22048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aacba1da5c1643d5aa7ddfa760c811d8" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" is_pre_election: true
I20260809 08:47:53.059913 21809 raft_consensus.cc:2454] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 0.
I20260809 08:47:53.059999 22048 raft_consensus.cc:2454] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 0.
I20260809 08:47:53.060124 22177 leader_election.cc:302] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5787992ac5be4a1d8e69fe2a8d63e995, c4dbbdabe71a486b8f7f1ab425f204bd; no voters:
I20260809 08:47:53.060283 22417 raft_consensus.cc:2781] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:53.060343 22417 raft_consensus.cc:482] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:53.060359 22417 raft_consensus.cc:3037] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:53.061401 22417 raft_consensus.cc:504] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:53.061517 22417 leader_election.cc:288] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 election: Requested vote from peers f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325), c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037)
I20260809 08:47:53.061667 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aacba1da5c1643d5aa7ddfa760c811d8" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
I20260809 08:47:53.061694 22048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aacba1da5c1643d5aa7ddfa760c811d8" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
I20260809 08:47:53.061758 22048 raft_consensus.cc:3037] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:53.061758 21809 raft_consensus.cc:3037] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:53.062896 22048 raft_consensus.cc:2454] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 1.
I20260809 08:47:53.063056 21809 raft_consensus.cc:2454] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 1.
I20260809 08:47:53.063097 22178 leader_election.cc:302] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5787992ac5be4a1d8e69fe2a8d63e995, f23d117eca2d4ddc8f8c786e5b39b616; no voters:
I20260809 08:47:53.063275 22417 raft_consensus.cc:2781] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:53.063393 22417 raft_consensus.cc:686] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 LEADER]: Becoming Leader. State: Replica: 5787992ac5be4a1d8e69fe2a8d63e995, State: Running, Role: LEADER
I20260809 08:47:53.063464 22417 consensus_queue.cc:237] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } }
I20260809 08:47:53.064307 20327 catalog_manager.cc:5495] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:53.064932 21533 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: NON_VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: true } } }
I20260809 08:47:53.065026 21533 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:53.065595 22427 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: attempting to promote NON_VOTER 7ea2b5c9b35148d699ccb3d838887741 to VOTER
I20260809 08:47:53.065729 22427 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } }
I20260809 08:47:53.066130 21278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 08:47:53.066162 21598 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 LEARNER]: Refusing update from remote peer 8a18dc7338874f0aadbbb3930f504831: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 08:47:53.066313 22427 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 08:47:53.066411 22428 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 08:47:53.067075 22428 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.067345 21278 raft_consensus.cc:2932] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.067306 21598 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068018 22428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068095 22428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:53.068148 22428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068202 22428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:53.068205 21532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068296 21532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:53.068409 22419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068490 22419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:53.068593 22433 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8a18dc7338874f0aadbbb3930f504831" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:53.068678 22433 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:53.086567 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.084662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43208 (local address 127.10.184.67:39085)
0809 08:47:53.084700 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.084702 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.084710 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:53.084744 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.084745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.084759 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.084817 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:53.084946 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.085387 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.086042 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.086215 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.086257 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.086350 (+ 93us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:53.086437 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:53.086491 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.086503 (+ 12us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.086519 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:47:53.088328 22048 raft_consensus.cc:1261] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Refusing update from remote peer 5787992ac5be4a1d8e69fe2a8d63e995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:53.088361 21809 raft_consensus.cc:1261] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Refusing update from remote peer 5787992ac5be4a1d8e69fe2a8d63e995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:53.088639 22425 consensus_queue.cc:1035] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:53.088757 22417 consensus_queue.cc:1035] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:53.093072 22439 mvcc.cc:205] Tried to move back new op lower bound from 7316542558567747584 to 7316542558468562944. Current Snapshot: MvccSnapshot[applied={T|T < 7316542558567747584}]
I20260809 08:47:53.093899 22435 mvcc.cc:205] Tried to move back new op lower bound from 7316542558567747584 to 7316542558468562944. Current Snapshot: MvccSnapshot[applied={T|T < 7316542558567747584 or (T in {7316542558567747584})}]
I20260809 08:47:53.094003 22438 mvcc.cc:205] Tried to move back new op lower bound from 7316542558567747584 to 7316542558468562944. Current Snapshot: MvccSnapshot[applied={T|T < 7316542558567747584}]
I20260809 08:47:53.095191 20326 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:53.095270 20326 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:53.096563 20326 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8a18dc7338874f0aadbbb3930f504831: Sending DeleteTablet for 3 replicas of tablet aacba1da5c1643d5aa7ddfa760c811d8
I20260809 08:47:53.096942 22245 tablet_service.cc:1514] Processing DeleteTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:53 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:43206
I20260809 08:47:53.096997 21774 tablet_service.cc:1514] Processing DeleteTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:53 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:35908
I20260809 08:47:53.097137 22011 tablet_service.cc:1514] Processing DeleteTablet for tablet aacba1da5c1643d5aa7ddfa760c811d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:53 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:57948
I20260809 08:47:53.097810 22455 tablet_replica.cc:330] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: stopping tablet replica
I20260809 08:47:53.097998 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 20274
I20260809 08:47:53.097921 22455 raft_consensus.cc:2229] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:53.109220 22455 raft_consensus.cc:2258] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:53.109782 22455 ts_tablet_manager.cc:1905] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:53.111832 22455 ts_tablet_manager.cc:1918] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:53.111989 22449 tablet_replica.cc:330] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: stopping tablet replica
I20260809 08:47:53.112129 22449 raft_consensus.cc:2229] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:53.112253 22449 raft_consensus.cc:2258] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:53.112730 22449 ts_tablet_manager.cc:1905] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:53.114686 22451 tablet_replica.cc:330] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: stopping tablet replica
I20260809 08:47:53.114818 22455 log.cc:1196] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/aacba1da5c1643d5aa7ddfa760c811d8
I20260809 08:47:53.114827 22451 raft_consensus.cc:2229] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:53.114950 22451 raft_consensus.cc:2258] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:53.115315 22455 ts_tablet_manager.cc:1939] T aacba1da5c1643d5aa7ddfa760c811d8 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting consensus metadata
W20260809 08:47:53.115726 21938 connection.cc:441] server connection from 127.0.0.1:57948 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:57948 (request call id 1) could send its response
I20260809 08:47:53.114707 22449 ts_tablet_manager.cc:1918] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:53.115973 22449 log.cc:1196] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/aacba1da5c1643d5aa7ddfa760c811d8
I20260809 08:47:53.116308 22449 ts_tablet_manager.cc:1939] T aacba1da5c1643d5aa7ddfa760c811d8 P c4dbbdabe71a486b8f7f1ab425f204bd: Deleting consensus metadata
W20260809 08:47:53.116689 21690 connection.cc:441] server connection from 127.0.0.1:35908 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:35908 (request call id 1) could send its response
I20260809 08:47:53.117223 22451 ts_tablet_manager.cc:1905] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:53.119503 22451 ts_tablet_manager.cc:1918] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:53.119567 22451 log.cc:1196] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/aacba1da5c1643d5aa7ddfa760c811d8
I20260809 08:47:53.120441 22451 ts_tablet_manager.cc:1939] T aacba1da5c1643d5aa7ddfa760c811d8 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting consensus metadata
W20260809 08:47:53.120859 22176 connection.cc:441] server connection from 127.0.0.1:43206 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:43206 (request call id 1) could send its response
I20260809 08:47:53.122198 22385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.097483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39458 (local address 127.10.184.124:43729)
0809 08:47:53.097569 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.097571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.109085 (+ 11514us) server_negotiation.cc:407] Connection header received
0809 08:47:53.109257 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.109258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.109269 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.109299 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:53.110298 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.111038 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.112088 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.112371 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.112418 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.115092 (+ 2674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:53.115094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:53.115094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:53.115102 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:53.116782 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.120020 (+ 3238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.120021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.120021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.120088 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.121634 (+ 1546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.121635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.121636 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.121767 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:53.121850 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.122042 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.122107 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 08:47:53.123822 21899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.115805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39470 (local address 127.10.184.124:43729)
0809 08:47:53.115843 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.115844 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.116961 (+ 1117us) server_negotiation.cc:407] Connection header received
0809 08:47:53.117054 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.117054 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.117064 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.117091 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:53.117842 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.118493 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.119501 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.119787 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.119831 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.121166 (+ 1335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:53.121167 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:53.121167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:53.121174 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:53.122871 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.123174 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.123175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.123175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.123222 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.123384 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.123385 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.123385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.123501 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:53.123580 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.123679 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.123744 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:53.124401 22389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:53.115485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33658 (local address 127.10.184.125:44471)
0809 08:47:53.115529 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:53.115531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:53.116153 (+ 622us) server_negotiation.cc:407] Connection header received
0809 08:47:53.116246 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:53.116247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:53.116260 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:53.116289 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:53.117043 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.117987 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.119082 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:53.119415 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:53.119454 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:53.120984 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:53.120986 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:53.120986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:53.120994 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:53.122883 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.123403 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.123405 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.123405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.123454 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:53.123662 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:53.123674 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:53.123674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:53.123807 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:53.123873 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:53.124271 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 08:47:53.124343 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:54.094353 22466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.094010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:56861)
0809 08:47:54.094178 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.094278 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":63,"threads_started":1}
W20260809 08:47:54.094585 22364 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:38185 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.094745 22468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.094186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:59649)
0809 08:47:54.094604 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.094701 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":60,"threads_started":1}
W20260809 08:47:54.095026 21892 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:38185 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.095126 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.094561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:59553)
0809 08:47:54.095010 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.095083 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":49,"threads_started":1}
W20260809 08:47:54.095314 22138 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:38185 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.597345 22471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a18dc7338874f0aadbbb3930f504831)
I20260809 08:47:54.597406 22472 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8a18dc7338874f0aadbbb3930f504831)
I20260809 08:47:54.597469 22471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } }
I20260809 08:47:54.597543 22472 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } }
I20260809 08:47:54.597714 22471 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185), e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125:44471)
I20260809 08:47:54.597813 22472 leader_election.cc:288] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185), 7ea2b5c9b35148d699ccb3d838887741 (127.10.184.124:43729)
I20260809 08:47:54.598120 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.597904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43878)
0809 08:47:54.598019 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.598076 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":58,"threads_started":1}
W20260809 08:47:54.598238 21236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.598505 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.597862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43872)
0809 08:47:54.598406 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.598465 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":72,"threads_started":1}
W20260809 08:47:54.598618 21569 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.598812 22481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43894)
0809 08:47:54.598733 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.598778 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260809 08:47:54.598956 21236 leader_election.cc:334] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185): Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.599041 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43900)
0809 08:47:54.598989 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.599016 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:47:54.599124 21569 leader_election.cc:334] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185): Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.600989 22479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33660 (local address 127.10.184.125:44471)
0809 08:47:54.598286 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:54.598288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:54.598338 (+ 50us) server_negotiation.cc:407] Connection header received
0809 08:47:54.598397 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:54.598398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:54.598408 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:54.598754 (+ 346us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:54.598792 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.599411 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600320 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.600683 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600742 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.600752 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:54.600867 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 08:47:54.600934 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":46,"threads_started":1}
I20260809 08:47:54.600986 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:44471 (local address 127.0.0.1:33660)
0809 08:47:54.598281 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.598292 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:54.598345 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:54.598510 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:54.598511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:54.598514 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:54.598678 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.598683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.599487 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.599488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.600244 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.600245 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.600750 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.600811 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.600815 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:54.600857 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:54.600920 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":52,"threads_started":1}
I20260809 08:47:54.601245 22487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39478 (local address 127.10.184.124:43729)
0809 08:47:54.598436 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:54.598438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:54.598515 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:47:54.598753 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:54.598754 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:54.598766 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:54.598815 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:54.599046 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.599573 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600479 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.600853 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600951 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.600960 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:54.601146 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 08:47:54.601196 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":66,"threads_started":1}
I20260809 08:47:54.601207 21278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ea2b5c9b35148d699ccb3d838887741" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" is_pre_election: true
I20260809 08:47:54.601274 22475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.598185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:43729 (local address 127.0.0.1:39478)
0809 08:47:54.598445 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.598455 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:54.598547 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:54.598887 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:54.598888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:54.598891 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:54.598956 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.598958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.599658 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.599659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.600394 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.600396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.600962 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.600963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.601026 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.601032 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:47:54.601123 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 08:47:54.601165 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":35,"threads_started":1}
I20260809 08:47:54.601308 21278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7ea2b5c9b35148d699ccb3d838887741 in term 1.
I20260809 08:47:54.601459 21569 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ea2b5c9b35148d699ccb3d838887741, e7d9573e9b324791a2a16d9e38ea00c8; no voters: 8a18dc7338874f0aadbbb3930f504831
I20260809 08:47:54.601562 22471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 08:47:54.601527 21598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7ea2b5c9b35148d699ccb3d838887741" is_pre_election: true
I20260809 08:47:54.601588 22471 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8a18dc7338874f0aadbbb3930f504831)
I20260809 08:47:54.601604 22471 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:47:54.602267 22471 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } }
I20260809 08:47:54.602387 22471 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 election: Requested vote from peers 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185), e7d9573e9b324791a2a16d9e38ea00c8 (127.10.184.125:44471)
I20260809 08:47:54.602394 21598 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e7d9573e9b324791a2a16d9e38ea00c8 in current term 2: Already voted for candidate 7ea2b5c9b35148d699ccb3d838887741 in this term.
I20260809 08:47:54.602588 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.602503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43910)
0809 08:47:54.602536 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.602567 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 08:47:54.602615 21235 leader_election.cc:302] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e7d9573e9b324791a2a16d9e38ea00c8; no voters: 7ea2b5c9b35148d699ccb3d838887741, 8a18dc7338874f0aadbbb3930f504831
I20260809 08:47:54.602658 21278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7ea2b5c9b35148d699ccb3d838887741" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "e7d9573e9b324791a2a16d9e38ea00c8"
I20260809 08:47:54.602742 22472 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:47:54.603003 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.602922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43912)
0809 08:47:54.602964 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.602988 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:47:54.603071 21569 leader_election.cc:334] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185): Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
I20260809 08:47:54.603478 22472 raft_consensus.cc:2726] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:47:54.604130 21278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7ea2b5c9b35148d699ccb3d838887741 in term 2.
I20260809 08:47:54.604295 21569 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7ea2b5c9b35148d699ccb3d838887741, e7d9573e9b324791a2a16d9e38ea00c8; no voters: 8a18dc7338874f0aadbbb3930f504831
I20260809 08:47:54.604400 22471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:47:54.604584 22471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [term 2 LEADER]: Becoming Leader. State: Replica: 7ea2b5c9b35148d699ccb3d838887741, State: Running, Role: LEADER
I20260809 08:47:54.604657 22471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } }
I20260809 08:47:54.604974 22491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea2b5c9b35148d699ccb3d838887741. Latest consensus state: current_term: 2 leader_uuid: "7ea2b5c9b35148d699ccb3d838887741" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:54.605070 22491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:54.605244 22495 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:54.605738 22495 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:54.605813 22495 catalog_manager.cc:1259] Loaded cluster ID: b19e39e8a5b34b578567f3562ade15e5
I20260809 08:47:54.605844 22495 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:54.606003 22495 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:54.606076 22495 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Loaded TSK: 0
I20260809 08:47:54.606211 22495 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:47:54.650269 21586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:54.651765 21586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:54.652688 21278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [term 2 FOLLOWER]: Refusing update from remote peer 7ea2b5c9b35148d699ccb3d838887741: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 08:47:54.652712 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.652598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43930)
0809 08:47:54.652641 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.652685 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 08:47:54.653015 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.652942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.0.0.1:43944)
0809 08:47:54.652973 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.652999 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 08:47:54.653046 22491 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 08:47:54.653182 21569 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 -> Peer 8a18dc7338874f0aadbbb3930f504831 (127.10.184.126:38185): Couldn't send request to peer 8a18dc7338874f0aadbbb3930f504831. Status: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 08:47:54.653635 22472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea2b5c9b35148d699ccb3d838887741. Latest consensus state: current_term: 2 leader_uuid: "7ea2b5c9b35148d699ccb3d838887741" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:54.653719 22472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:54.654091 22491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "7ea2b5c9b35148d699ccb3d838887741" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:54.654172 22491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:54.654340 22472 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "7ea2b5c9b35148d699ccb3d838887741" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8a18dc7338874f0aadbbb3930f504831" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38185 } } peers { permanent_uuid: "e7d9573e9b324791a2a16d9e38ea00c8" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 44471 } attrs { promote: false } } peers { permanent_uuid: "7ea2b5c9b35148d699ccb3d838887741" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 43729 } attrs { promote: false } } }
I20260809 08:47:54.654424 22472 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d9573e9b324791a2a16d9e38ea00c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:47:54.654484 22497 mvcc.cc:205] Tried to move back new op lower bound from 7316542564975722496 to 7316542564781252608. Current Snapshot: MvccSnapshot[applied={T|T < 7316542564975722496 or (T in {7316542564975722496})}]
I20260809 08:47:54.658918 22473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.655835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:41325 (local address 127.0.0.1:57964)
0809 08:47:54.655989 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.655997 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:54.656019 (+ 22us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:54.656204 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:54.656205 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:54.656206 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:54.656254 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.656255 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.657021 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.657022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.658007 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.658009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.658678 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.658679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.658740 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.658745 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:54.658817 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:54.658855 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260809 08:47:54.658941 22509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.655992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57964 (local address 127.10.184.66:41325)
0809 08:47:54.656096 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:54.656097 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:54.656104 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:54.656126 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:54.656127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:54.656136 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:54.656193 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:54.656314 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.656953 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.658096 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.658603 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.658674 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.658687 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:54.658826 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 08:47:54.658869 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84,"thread_start_us":61,"threads_started":1}
I20260809 08:47:54.659255 22011 tablet_service.cc:1467] Processing CreateTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 (DEFAULT_TABLE table=test-table [id=70d8fc8a33e24da69f340b8ec12cb527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:54.659354 22476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.655799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39085 (local address 127.0.0.1:43222)
0809 08:47:54.655866 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.655879 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:54.655934 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:54.656374 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:54.656374 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:54.656377 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:54.656419 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.656420 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.657331 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.657332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.658382 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.658385 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.659132 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.659134 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.659209 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.659212 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:47:54.659257 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:47:54.659294 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":28}
I20260809 08:47:54.659392 22011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f51f4394b64ddd8533d8b401aac2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:54.659565 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.655864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.10.184.67:39085)
0809 08:47:54.656241 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:54.656243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:54.656251 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:54.656275 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:54.656275 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:54.656287 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:54.656474 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:54.656545 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.657258 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.658471 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.659052 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.659117 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.659128 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:54.659449 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 08:47:54.659501 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":68,"threads_started":1}
I20260809 08:47:54.659772 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.655877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:45037 (local address 127.0.0.1:35924)
0809 08:47:54.656091 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.656099 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:54.656130 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:54.656341 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:54.656342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:54.656343 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:54.656373 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.656374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.657326 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.657327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.658179 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:54.658182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.659562 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.659563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:54.659619 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.659623 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:54.659676 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:47:54.659712 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":22,"thread_start_us":66,"threads_started":1}
I20260809 08:47:54.659765 22245 tablet_service.cc:1467] Processing CreateTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 (DEFAULT_TABLE table=test-table [id=70d8fc8a33e24da69f340b8ec12cb527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:54.659797 22513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.656112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35924 (local address 127.10.184.65:45037)
0809 08:47:54.656242 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:54.656244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:54.656251 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:54.656274 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:54.656275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:54.656284 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:54.656342 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:54.656451 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.657233 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.658523 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:54.659389 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:54.659527 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:54.659540 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:54.659694 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:47:54.659738 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260809 08:47:54.659879 22245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f51f4394b64ddd8533d8b401aac2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:54.661262 22525 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Bootstrap starting.
I20260809 08:47:54.661391 21774 tablet_service.cc:1467] Processing CreateTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 (DEFAULT_TABLE table=test-table [id=70d8fc8a33e24da69f340b8ec12cb527]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:54.661490 21774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d5f51f4394b64ddd8533d8b401aac2b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:54.661592 22525 tablet_bootstrap.cc:654] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:54.662153 22525 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: No bootstrap required, opened a new log
I20260809 08:47:54.662196 22525 ts_tablet_manager.cc:1397] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:54.662286 22525 raft_consensus.cc:348] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.662329 22525 raft_consensus.cc:374] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:54.662343 22525 raft_consensus.cc:729] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5787992ac5be4a1d8e69fe2a8d63e995, State: Initialized, Role: FOLLOWER
I20260809 08:47:54.662379 22525 consensus_queue.cc:260] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.662496 22525 ts_tablet_manager.cc:1428] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:54.662583 22529 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: Bootstrap starting.
I20260809 08:47:54.662963 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.662694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:35685)
0809 08:47:54.662892 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.662929 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
I20260809 08:47:54.663049 22529 tablet_bootstrap.cc:654] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:54.663125 22514 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Bootstrap starting.
W20260809 08:47:54.663147 22368 tablet.cc:2369] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:54.663605 22514 tablet_bootstrap.cc:654] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:54.663728 22529 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: No bootstrap required, opened a new log
I20260809 08:47:54.663802 22529 ts_tablet_manager.cc:1397] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:54.663910 22529 raft_consensus.cc:348] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.663981 22529 raft_consensus.cc:374] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:54.664003 22529 raft_consensus.cc:729] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4dbbdabe71a486b8f7f1ab425f204bd, State: Initialized, Role: FOLLOWER
I20260809 08:47:54.664065 22529 consensus_queue.cc:260] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.664213 22529 ts_tablet_manager.cc:1428] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:54.664464 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.664023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:39057)
0809 08:47:54.664390 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.664422 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351}
I20260809 08:47:54.664640 22514 tablet_bootstrap.cc:492] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: No bootstrap required, opened a new log
I20260809 08:47:54.664669 21890 heartbeater.cc:503] Master 127.10.184.124:43729 was elected leader, sending a full tablet report...
I20260809 08:47:54.664697 22514 ts_tablet_manager.cc:1397] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:54.664844 22514 raft_consensus.cc:348] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.664963 22514 raft_consensus.cc:374] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:54.664989 22514 raft_consensus.cc:729] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f23d117eca2d4ddc8f8c786e5b39b616, State: Initialized, Role: FOLLOWER
I20260809 08:47:54.665040 22514 consensus_queue.cc:260] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.665189 22514 ts_tablet_manager.cc:1428] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:54.665385 22537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.664494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:37223)
0809 08:47:54.665315 (+ 821us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.665354 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":70,"threads_started":1}
I20260809 08:47:54.665762 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.665571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:54371)
0809 08:47:54.665679 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.665732 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":72,"threads_started":1}
I20260809 08:47:54.665817 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.665419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:56833)
0809 08:47:54.665750 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.665780 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":87,"threads_started":1}
I20260809 08:47:54.665812 22137 heartbeater.cc:503] Master 127.10.184.124:43729 was elected leader, sending a full tablet report...
I20260809 08:47:54.666172 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.666082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:46389)
0809 08:47:54.666118 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.666148 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 08:47:54.868772 21894 tablet.cc:2369] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:54.935387 22528 raft_consensus.cc:482] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:54.935475 22528 raft_consensus.cc:504] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.935642 22528 leader_election.cc:288] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037), f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
I20260809 08:47:54.935818 22048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f51f4394b64ddd8533d8b401aac2b4" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" is_pre_election: true
I20260809 08:47:54.935845 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f51f4394b64ddd8533d8b401aac2b4" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" is_pre_election: true
I20260809 08:47:54.935925 21809 raft_consensus.cc:2454] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 0.
I20260809 08:47:54.935927 22048 raft_consensus.cc:2454] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 0.
I20260809 08:47:54.936088 22177 leader_election.cc:302] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5787992ac5be4a1d8e69fe2a8d63e995, c4dbbdabe71a486b8f7f1ab425f204bd; no voters:
I20260809 08:47:54.936200 22528 raft_consensus.cc:2781] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:54.936235 22528 raft_consensus.cc:482] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:54.936259 22528 raft_consensus.cc:3037] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:54.937006 22528 raft_consensus.cc:504] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.937129 22528 leader_election.cc:288] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 election: Requested vote from peers c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037), f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325)
I20260809 08:47:54.937295 22048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f51f4394b64ddd8533d8b401aac2b4" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f23d117eca2d4ddc8f8c786e5b39b616"
I20260809 08:47:54.937304 21809 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d5f51f4394b64ddd8533d8b401aac2b4" candidate_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd"
I20260809 08:47:54.937362 21809 raft_consensus.cc:3037] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:54.937362 22048 raft_consensus.cc:3037] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:54.937911 21809 raft_consensus.cc:2454] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 1.
I20260809 08:47:54.937981 22048 raft_consensus.cc:2454] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5787992ac5be4a1d8e69fe2a8d63e995 in term 1.
I20260809 08:47:54.938068 22177 leader_election.cc:302] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5787992ac5be4a1d8e69fe2a8d63e995, c4dbbdabe71a486b8f7f1ab425f204bd; no voters:
I20260809 08:47:54.938160 22528 raft_consensus.cc:2781] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:54.938305 22528 raft_consensus.cc:686] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 LEADER]: Becoming Leader. State: Replica: 5787992ac5be4a1d8e69fe2a8d63e995, State: Running, Role: LEADER
I20260809 08:47:54.938378 22363 heartbeater.cc:503] Master 127.10.184.124:43729 was elected leader, sending a full tablet report...
I20260809 08:47:54.938375 22528 consensus_queue.cc:237] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } }
I20260809 08:47:54.938683 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:54.938569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:38279)
0809 08:47:54.938610 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:54.938658 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 08:47:54.939101 21586 catalog_manager.cc:5495] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5787992ac5be4a1d8e69fe2a8d63e995" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:55.008883 22504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.006771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43236 (local address 127.10.184.67:39085)
0809 08:47:55.006826 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:55.006829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:55.006835 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:47:55.006859 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:55.006860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:55.006870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:55.006920 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:55.007038 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:55.007526 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:55.008258 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:55.008460 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:55.008513 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:55.008649 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:55.008687 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:55.008728 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:55.008784 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 08:47:55.008808 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:47:55.009773 21809 raft_consensus.cc:1261] T d5f51f4394b64ddd8533d8b401aac2b4 P c4dbbdabe71a486b8f7f1ab425f204bd [term 1 FOLLOWER]: Refusing update from remote peer 5787992ac5be4a1d8e69fe2a8d63e995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:55.009799 22048 raft_consensus.cc:1261] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Refusing update from remote peer 5787992ac5be4a1d8e69fe2a8d63e995: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:55.010018 22542 consensus_queue.cc:1035] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4dbbdabe71a486b8f7f1ab425f204bd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 45037 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:55.010130 22528 consensus_queue.cc:1035] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f23d117eca2d4ddc8f8c786e5b39b616" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 41325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:55.010995 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.010879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:35027)
0809 08:47:55.010929 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.010963 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:47:55.011680 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.010909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:37403)
0809 08:47:55.011511 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.011648 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":582}
I20260809 08:47:55.011966 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.011876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:33427)
0809 08:47:55.011909 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.011939 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260809 08:47:55.012619 22540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.012532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.65:40825)
0809 08:47:55.012562 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.012590 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 08:47:55.013255 21587 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:55.013355 21587 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:55.013435 22530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.013333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.67:60135)
0809 08:47:55.013389 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.013412 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 08:47:55.014081 22539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.013986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38185 (local address 127.10.184.66:59793)
0809 08:47:55.014022 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.014054 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:38185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 08:47:55.014700 21587 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ea2b5c9b35148d699ccb3d838887741: Sending DeleteTablet for 3 replicas of tablet d5f51f4394b64ddd8533d8b401aac2b4
I20260809 08:47:55.014895 22011 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:55 UTC) from {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:57964
I20260809 08:47:55.015002 22245 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:55 UTC) from {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:43222
I20260809 08:47:55.015002 21774 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:55 UTC) from {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:35924
I20260809 08:47:55.015205 22557 tablet_replica.cc:330] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: stopping tablet replica
I20260809 08:47:55.015206 22011 tablet_service.cc:1514] Processing DeleteTablet for tablet d5f51f4394b64ddd8533d8b401aac2b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:55 UTC) from {username='oryx', principal='oryx/127.10.184.124@KRBTEST.COM'} at 127.0.0.1:57964
I20260809 08:47:55.015350 22558 tablet_replica.cc:330] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: stopping tablet replica
I20260809 08:47:55.015394 22558 raft_consensus.cc:2229] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:55.015317 22557 raft_consensus.cc:2229] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:55.015458 22558 raft_consensus.cc:2258] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:55.015461 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 21529
I20260809 08:47:55.015534 22557 raft_consensus.cc:2258] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:55.019420 22558 ts_tablet_manager.cc:1905] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:55.021420 22558 ts_tablet_manager.cc:1918] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:55.021482 22558 log.cc:1196] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/d5f51f4394b64ddd8533d8b401aac2b4
I20260809 08:47:55.021768 22558 ts_tablet_manager.cc:1939] T d5f51f4394b64ddd8533d8b401aac2b4 P 5787992ac5be4a1d8e69fe2a8d63e995: Deleting consensus metadata
I20260809 08:47:55.022115 21566 catalog_manager.cc:4841] TS 5787992ac5be4a1d8e69fe2a8d63e995 (127.10.184.67:39085): tablet d5f51f4394b64ddd8533d8b401aac2b4 (table test-table [id=70d8fc8a33e24da69f340b8ec12cb527]) successfully deleted
I20260809 08:47:55.027081 22557 ts_tablet_manager.cc:1905] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:55.028373 22557 ts_tablet_manager.cc:1918] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:55.028425 22557 log.cc:1196] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/d5f51f4394b64ddd8533d8b401aac2b4
I20260809 08:47:55.028609 22557 ts_tablet_manager.cc:1939] T d5f51f4394b64ddd8533d8b401aac2b4 P f23d117eca2d4ddc8f8c786e5b39b616: Deleting consensus metadata
I20260809 08:47:55.028926 21569 catalog_manager.cc:4841] TS f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325): tablet d5f51f4394b64ddd8533d8b401aac2b4 (table test-table [id=70d8fc8a33e24da69f340b8ec12cb527]) successfully deleted
W20260809 08:47:55.029084 21569 catalog_manager.cc:4808] TS f23d117eca2d4ddc8f8c786e5b39b616 (127.10.184.66:41325): delete failed for tablet d5f51f4394b64ddd8533d8b401aac2b4 because the tablet was not found. No further retry: Not found: Tablet not found: d5f51f4394b64ddd8533d8b401aac2b4
I20260809 08:47:55.032219 22507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:55.032124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:45037 (local address 127.0.0.1:35934)
0809 08:47:55.032172 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:55.032194 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:45037: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:47:55.032266 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 21910
W20260809 08:47:55.032362 21566 catalog_manager.cc:4584] TS c4dbbdabe71a486b8f7f1ab425f204bd (127.10.184.65:45037): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d5f51f4394b64ddd8533d8b401aac2b4: Network error: Client connection negotiation failed: client connection to 127.10.184.65:45037: connect: Connection refused (error 111)
I20260809 08:47:55.039652 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 22148
I20260809 08:47:55.051683 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 21053
I20260809 08:47:55.060225 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 21357
2026-08-09T08:47:55Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5030 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:55 dist-test-slave-ncp9 krb5kdc[22567](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:55 dist-test-slave-ncp9 krb5kdc[22567](info): set up 2 sockets
Aug 09 08:47:55 dist-test-slave-ncp9 krb5kdc[22567](info): commencing operation
krb5kdc: starting...
W20260809 08:47:56.188314 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:56Z Disabled control of system clock
WARNING: no policy specified for oryx/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab.
Entry for principal oryx/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab.
I20260809 08:47:56.297272 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36159
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab
--principal=oryx/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:36159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:56.375254 22584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:56.375447 22584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:56.375476 22584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:56.376585 22584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:56.376626 22584 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 08:47:56.376644 22584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:56.376662 22584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:56.376685 22584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:56.376709 22584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:56.378218 22584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:36159
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.126.keytab
--principal=oryx/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:56.378458 22584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:56.378643 22584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:56.383340 22592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.383343 22590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.383387 22584 server_base.cc:1034] running on GCE node
W20260809 08:47:56.383388 22594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.383800 22584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:56.384063 22584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:56.385200 22584 hybrid_clock.cc:648] HybridClock initialized: now 1786265276385189 us; error 27 us; skew 500 ppm
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:56.387457 22584 init.cc:377] Logged in from keytab as oryx/127.10.184.126@KRBTEST.COM (short username oryx)
I20260809 08:47:56.387871 22584 webserver.cc:460] Webserver started at http://127.10.184.126:40467/ using document root <none> and password file <none>
I20260809 08:47:56.388077 22584 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:56.388123 22584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:56.388252 22584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:56.389299 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "79d76f9fa0254d9fbb81c651dd42b0c1"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "05549eb926bc8fed79983916d4630c7d"
server_key_iv: "58d9d7a989d794b69f6dad66a07e0849"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.389623 22584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "79d76f9fa0254d9fbb81c651dd42b0c1"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "05549eb926bc8fed79983916d4630c7d"
server_key_iv: "58d9d7a989d794b69f6dad66a07e0849"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.390873 22584 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:56.391505 22605 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.391652 22584 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:56.391739 22584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "79d76f9fa0254d9fbb81c651dd42b0c1"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "05549eb926bc8fed79983916d4630c7d"
server_key_iv: "58d9d7a989d794b69f6dad66a07e0849"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.391798 22584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:56.403383 22584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:56.404932 22584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:56.405058 22584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:56.409363 22584 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:36159
I20260809 08:47:56.409423 22707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:36159 every 8 connection(s)
I20260809 08:47:56.409693 22584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:47:56.410307 22709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:56.410871 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 22584
I20260809 08:47:56.410966 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:47:56.411130 10977 external_mini_cluster.cc:1428] Setting key 2f7eb4930c96a5c753b2133cfe492657
I20260809 08:47:56.413177 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Bootstrap starting.
I20260809 08:47:56.413746 22709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:56.413993 22709 log.cc:824] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:56.414569 22709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: No bootstrap required, opened a new log
I20260809 08:47:56.416352 22709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } }
I20260809 08:47:56.416481 22709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:56.416512 22709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d76f9fa0254d9fbb81c651dd42b0c1, State: Initialized, Role: FOLLOWER
I20260809 08:47:56.416656 22709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } }
I20260809 08:47:56.416750 22709 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:47:56.416781 22709 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:47:56.416826 22709 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:56.417487 22709 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } }
I20260809 08:47:56.417585 22709 leader_election.cc:302] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79d76f9fa0254d9fbb81c651dd42b0c1; no voters:
I20260809 08:47:56.417770 22709 leader_election.cc:288] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:47:56.417816 22719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:56.417984 22719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [term 1 LEADER]: Becoming Leader. State: Replica: 79d76f9fa0254d9fbb81c651dd42b0c1, State: Running, Role: LEADER
I20260809 08:47:56.418012 22709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:47:56.418112 22719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } }
I20260809 08:47:56.418742 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d76f9fa0254d9fbb81c651dd42b0c1. Latest consensus state: current_term: 1 leader_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } } }
I20260809 08:47:56.418752 22724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79d76f9fa0254d9fbb81c651dd42b0c1" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 36159 } } }
I20260809 08:47:56.418838 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [sys.catalog]: This master's current role is: LEADER
I20260809 08:47:56.418850 22724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1 [sys.catalog]: This master's current role is: LEADER
W20260809 08:47:56.419974 22745 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:47:56.420112 22745 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:47:56.421317 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.411691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.10.184.126:36159)
0809 08:47:56.411837 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.411846 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.411869 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:47:56.412264 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.412270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.412314 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.412421 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.413212 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.414154 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.414902 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.415304 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.415406 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.417677 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.417681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.417681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.417693 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.419795 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.420166 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.420169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.420170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.420271 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.420459 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.420460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.420460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.420757 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.420883 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.420997 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.421131 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":64,"threads_started":1}
I20260809 08:47:56.425028 22750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:47:56.425227 22750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:47:56.426661 22750 catalog_manager.cc:1347] Generated new cluster ID: 07d2daa6d02c4c408d6c42d05ceca3d6
I20260809 08:47:56.426716 22750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:47:56.437500 22750 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:47:56.438171 22750 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:47:56.443013 22750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Generated new TSK 0
I20260809 08:47:56.443162 22750 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
I20260809 08:47:56.469002 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36159
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:56.547935 22755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:56.548132 22755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:56.548164 22755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:56.549304 22755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:56.549342 22755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:56.549369 22755 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:56.549391 22755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:47:56.550855 22755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:56.551075 22755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:56.551285 22755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:56.555958 22763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.555958 22766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.556123 22755 server_base.cc:1034] running on GCE node
W20260809 08:47:56.556198 22764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.556398 22755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:56.556614 22755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:56.557744 22755 hybrid_clock.cc:648] HybridClock initialized: now 1786265276557732 us; error 27 us; skew 500 ppm
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:56.560079 22755 init.cc:377] Logged in from keytab as oryx/127.10.184.65@KRBTEST.COM (short username oryx)
I20260809 08:47:56.560487 22755 webserver.cc:460] Webserver started at http://127.10.184.65:34873/ using document root <none> and password file <none>
I20260809 08:47:56.560679 22755 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:56.560722 22755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:56.560846 22755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:56.561913 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "059ce69a2989456c8bfcd4909ebffe9b"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "c78db61a1249949d5a345344cd65435f"
server_key_iv: "c5319d211041216c486485ece6f0f536"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.562335 22755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "059ce69a2989456c8bfcd4909ebffe9b"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "c78db61a1249949d5a345344cd65435f"
server_key_iv: "c5319d211041216c486485ece6f0f536"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.564113 22755 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 08:47:56.564960 22779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.565126 22755 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:56.565227 22755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "059ce69a2989456c8bfcd4909ebffe9b"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "c78db61a1249949d5a345344cd65435f"
server_key_iv: "c5319d211041216c486485ece6f0f536"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.565302 22755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:56.580271 22755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:56.582250 22755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:56.582351 22755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:56.582803 22755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:56.582842 22755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.582861 22755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:56.582873 22755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:56.589701 22755 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:37321
I20260809 08:47:56.589776 22955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:37321 every 8 connection(s)
I20260809 08:47:56.590147 22755 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:47:56.591882 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.583165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:34405 (local address 127.10.184.126:36159)
0809 08:47:56.583228 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.583230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.583458 (+ 228us) server_negotiation.cc:407] Connection header received
0809 08:47:56.584052 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.584053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.584065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.584117 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.584953 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.585463 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.586438 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.586643 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.586704 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.589518 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.589520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.589520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.589528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.590891 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.591220 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.591220 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.591221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.591267 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.591424 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.591424 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.591424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.591515 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.591558 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.591755 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.591820 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:47:56.592006 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.583128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.65:34405)
0809 08:47:56.583324 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.583370 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.583946 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.584201 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.584204 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.584538 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.584810 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.584816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.585550 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.585551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.586358 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.586361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.586718 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.586720 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.586841 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.587296 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.587306 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.589432 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.590957 (+ 1525us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.590960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.590962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.591160 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.591302 (+ 142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.591303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.591303 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.591385 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.591556 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.591558 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.591658 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.591736 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.591845 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":79,"threads_started":1}
I20260809 08:47:56.592443 22956 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36159
I20260809 08:47:56.592525 22956 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:56.592694 22956 heartbeater.cc:511] Master 127.10.184.126:36159 requested a full tablet report, sending...
I20260809 08:47:56.593199 22634 ts_manager.cc:194] Registered new tserver with Master: 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321)
I20260809 08:47:56.593501 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 22755
I20260809 08:47:56.593614 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:47:56.593811 10977 external_mini_cluster.cc:1428] Setting key eda79c303863beb7701e796ee74f6975
I20260809 08:47:56.594072 22634 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:34405
I20260809 08:47:56.598745 22964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.593425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.65:59269)
0809 08:47:56.593577 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.593587 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.593665 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.593772 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.593773 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.593956 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.594018 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.594020 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.594784 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.594785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.595370 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.595372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.595702 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.595703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.595767 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.596115 (+ 348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.596120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.596367 (+ 247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.597842 (+ 1475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.597843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.597843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.597996 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.598180 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.598181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.598181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.598225 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.598471 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.598472 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.598514 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.598565 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.598650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":65,"threads_started":1}
I20260809 08:47:56.598767 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.593485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:59269 (local address 127.10.184.126:36159)
0809 08:47:56.593522 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.593524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.593632 (+ 108us) server_negotiation.cc:407] Connection header received
0809 08:47:56.593720 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.593720 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.593729 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.593769 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.594107 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.594699 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.595432 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.595643 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.595696 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.596446 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.596447 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.596447 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.596454 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.597767 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.598059 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.598060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.598061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.598118 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.598275 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.598276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.598277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.598408 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.598478 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.598609 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.598682 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for oryx/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
I20260809 08:47:56.617333 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36159
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:56.695747 22968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:56.695935 22968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:56.695966 22968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:56.697088 22968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:56.697125 22968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:56.697151 22968 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:56.697175 22968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:47:56.698684 22968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:56.698917 22968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:56.699126 22968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:56.703897 22976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.703907 22975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.703902 22978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.703962 22968 server_base.cc:1034] running on GCE node
I20260809 08:47:56.704263 22968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:56.704490 22968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:56.705621 22968 hybrid_clock.cc:648] HybridClock initialized: now 1786265276705609 us; error 24 us; skew 500 ppm
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:56.707911 22968 init.cc:377] Logged in from keytab as oryx/127.10.184.66@KRBTEST.COM (short username oryx)
I20260809 08:47:56.708343 22968 webserver.cc:460] Webserver started at http://127.10.184.66:33293/ using document root <none> and password file <none>
I20260809 08:47:56.708552 22968 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:56.708595 22968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:56.708717 22968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:56.709761 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "9bdeb752d89541308553e5f5731f0734"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "eddee590ecf58d13d3bf80f8c68a87dc"
server_key_iv: "7c00721197002307d6afda31de1fbfcb"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.710078 22968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "9bdeb752d89541308553e5f5731f0734"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "eddee590ecf58d13d3bf80f8c68a87dc"
server_key_iv: "7c00721197002307d6afda31de1fbfcb"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.711298 22968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:47:56.711932 22991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.712066 22968 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:47:56.712152 22968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "9bdeb752d89541308553e5f5731f0734"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "eddee590ecf58d13d3bf80f8c68a87dc"
server_key_iv: "7c00721197002307d6afda31de1fbfcb"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.712219 22968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:56.741089 22968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:56.742812 22968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:56.742936 22968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:56.743440 22968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:56.743481 22968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.743502 22968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:56.743513 22968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:56.750237 22968 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:33061
I20260809 08:47:56.750370 23181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:33061 every 8 connection(s)
I20260809 08:47:56.750622 22968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:47:56.751474 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 22968
I20260809 08:47:56.751587 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:47:56.751796 10977 external_mini_cluster.cc:1428] Setting key c7f4cfbac6dfa739f995aad2eca0adf6
I20260809 08:47:56.752709 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.743882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:42741 (local address 127.10.184.126:36159)
0809 08:47:56.743950 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.743953 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.744117 (+ 164us) server_negotiation.cc:407] Connection header received
0809 08:47:56.744587 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.744588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.744601 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.744683 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.745369 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.745867 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.746856 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.747084 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.747137 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.749857 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.749858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.749859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.749865 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.751295 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.751748 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.751748 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.751749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.751805 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.752087 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.752088 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.752088 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.752214 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.752279 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.752541 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.752621 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:47:56.752912 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.743824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.66:42741)
0809 08:47:56.743994 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.744034 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.744497 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.744759 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.744763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.745084 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.745272 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.745277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.745959 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.745961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.746769 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.746772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.747164 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.747165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.747284 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.747723 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.747732 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.749773 (+ 2041us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.751365 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.751370 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.751372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.751663 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.751896 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.751897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.751897 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.752029 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.752271 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.752276 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.752405 (+ 129us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.752512 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.752684 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":88,"threads_started":1}
I20260809 08:47:56.754344 23183 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36159
I20260809 08:47:56.754428 23183 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:56.754590 23183 heartbeater.cc:511] Master 127.10.184.126:36159 requested a full tablet report, sending...
I20260809 08:47:56.754907 22634 ts_manager.cc:194] Registered new tserver with Master: 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061)
I20260809 08:47:56.755337 22634 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:42741
I20260809 08:47:56.759764 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.754453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:49705 (local address 127.10.184.126:36159)
0809 08:47:56.754488 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.754490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.754702 (+ 212us) server_negotiation.cc:407] Connection header received
0809 08:47:56.754758 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.754759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.754769 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.754819 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.755179 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.755716 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.756606 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.756791 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.756843 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.757644 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.757645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.757645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.757662 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.758792 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.759074 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.759075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.759075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.759123 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.759281 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.759282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.759282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.759412 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.759467 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.759607 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.759683 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:47:56.759773 23193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.754495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.66:49705)
0809 08:47:56.754639 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.754650 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.754711 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.754862 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.754863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.755043 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.755099 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.755101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.755797 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.755798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.756530 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.756532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.756849 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.756850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.756929 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.757321 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.757325 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.757567 (+ 242us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.758860 (+ 1293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.758861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.758862 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.759016 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.759177 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.759178 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.759178 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.759232 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.759470 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.759471 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.759512 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.759577 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.759669 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":79,"threads_started":1}
WARNING: no policy specified for oryx/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
I20260809 08:47:56.775334 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:36159
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:56.855326 23195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:56.855501 23195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:56.855523 23195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:56.856580 23195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:56.856612 23195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:56.856630 23195 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 08:47:56.856648 23195 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:47:56.858119 23195 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.10.184.126:36159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:56.858289 23195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:56.858451 23195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:56.863128 23203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.863140 23200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:56.863169 23206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:56.863358 23195 server_base.cc:1034] running on GCE node
I20260809 08:47:56.863547 23195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:56.863778 23195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:56.864920 23195 hybrid_clock.cc:648] HybridClock initialized: now 1786265276864905 us; error 27 us; skew 500 ppm
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:47:56.867251 23195 init.cc:377] Logged in from keytab as oryx/127.10.184.67@KRBTEST.COM (short username oryx)
I20260809 08:47:56.867635 23195 webserver.cc:460] Webserver started at http://127.10.184.67:34323/ using document root <none> and password file <none>
I20260809 08:47:56.867848 23195 fs_manager.cc:359] Metadata directory not provided
I20260809 08:47:56.867894 23195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:47:56.868012 23195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:47:56.869033 23195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "1ed79483d3344acfb4a3e0c6693d015a"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "d63048d858f3ae35552cb02875545ebe"
server_key_iv: "64206b4fa96cfb2a812880dd4e04edff"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.869354 23195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "1ed79483d3344acfb4a3e0c6693d015a"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "d63048d858f3ae35552cb02875545ebe"
server_key_iv: "64206b4fa96cfb2a812880dd4e04edff"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.870675 23195 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:47:56.871425 23219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.871579 23195 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:56.871654 23195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "1ed79483d3344acfb4a3e0c6693d015a"
format_stamp: "Formatted at 2026-08-09 08:47:56 on dist-test-slave-ncp9"
server_key: "d63048d858f3ae35552cb02875545ebe"
server_key_iv: "64206b4fa96cfb2a812880dd4e04edff"
server_key_version: "encryptionkey@0"
I20260809 08:47:56.871721 23195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:47:56.890795 23195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:47:56.892505 23195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:47:56.892642 23195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:47:56.893208 23195 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:47:56.893247 23195 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.893275 23195 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:47:56.893292 23195 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:56.899474 23195 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:34783
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:56.899552 23400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:34783 every 8 connection(s)
I20260809 08:47:56.899849 23195 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:47:56.902530 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.893516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59851 (local address 127.10.184.126:36159)
0809 08:47:56.893578 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.893581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.893816 (+ 235us) server_negotiation.cc:407] Connection header received
0809 08:47:56.894288 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.894289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.894302 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.894378 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.895136 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.895896 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.897382 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.897682 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.897751 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.900130 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.900132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.900132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.900140 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.901478 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.901843 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.901843 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.901844 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.901897 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.902072 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.902072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.902073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.902166 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.902223 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.902410 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.902468 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:47:56.902696 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.893527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.67:59851)
0809 08:47:56.893686 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.893726 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.894196 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.894452 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.894456 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.894775 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.895038 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.895045 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.896000 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.896001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.897292 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.897295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.897774 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.897775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.897910 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.898337 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.898347 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.900026 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.901559 (+ 1533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.901563 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.901565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.901783 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.901944 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.901945 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.901945 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.902030 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.902217 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.902219 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.902320 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.902392 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.902501 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":73,"threads_started":1}
I20260809 08:47:56.903246 23401 heartbeater.cc:347] Connected to a master server at 127.10.184.126:36159
I20260809 08:47:56.903328 23401 heartbeater.cc:464] Registering TS with master...
I20260809 08:47:56.903505 23401 heartbeater.cc:511] Master 127.10.184.126:36159 requested a full tablet report, sending...
I20260809 08:47:56.903918 22636 ts_manager.cc:194] Registered new tserver with Master: 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:56.904392 22636 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:59851
I20260809 08:47:56.908608 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.904271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:36243 (local address 127.10.184.126:36159)
0809 08:47:56.904305 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.904307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.904470 (+ 163us) server_negotiation.cc:407] Connection header received
0809 08:47:56.904523 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.904524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.904534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.904591 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.904929 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.905383 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.906013 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.906192 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.906238 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.906895 (+ 657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.906896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.906897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.906902 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.907870 (+ 968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.908137 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.908137 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.908137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.908186 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.908298 (+ 112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.908298 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.908298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.908381 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.908418 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.908506 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.908556 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:47:56.908649 23412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.904216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.10.184.67:36243)
0809 08:47:56.904409 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.904419 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.904471 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.904594 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.904594 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.904770 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:56.904835 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.904837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.905460 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.905461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.905957 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.905959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.906243 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.906243 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.906288 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.906587 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:56.906591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:56.906831 (+ 240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:56.907931 (+ 1100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.907932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.907932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.908084 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.908220 (+ 136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:56.908220 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:56.908221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:56.908264 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:56.908414 (+ 150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:56.908415 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:56.908447 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.908488 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.908553 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":148,"threads_started":1}
I20260809 08:47:56.909380 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 23195
I20260809 08:47:56.909497 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:47:56.909682 10977 external_mini_cluster.cc:1428] Setting key fc1a62f272d9841f7f069a025f7e7494
I20260809 08:47:56.910455 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:47:56.930938 22716 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:56.926380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38754 (local address 127.10.184.126:36159)
0809 08:47:56.926430 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.926432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.926456 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:47:56.926479 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.926480 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.926488 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.926524 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.926999 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.927486 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.928132 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.928308 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.928350 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.930857 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38754: BlockingRecv error: recv got EOF from 127.0.0.1:38754 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 08:47:56 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:56.937573 22716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.931736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.10.184.126:36159)
0809 08:47:56.931790 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.931792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.931837 (+ 45us) server_negotiation.cc:407] Connection header received
0809 08:47:56.931897 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.931898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.931907 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.931948 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:56.932363 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.932807 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.933496 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.933678 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.933730 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.935560 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:56.935562 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:56.935562 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:56.935569 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:56.936585 (+ 1016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.936879 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.936880 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.936880 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.936962 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:56.937123 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:56.937124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:56.937124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:56.937233 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:56.937315 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.937415 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.937489 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:47:56.938841 22636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:56.940316 22636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:56.946683 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:33061 (local address 127.0.0.1:60448)
0809 08:47:56.942466 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.942503 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.942566 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.943175 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.943176 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.943179 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:56.943237 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.943239 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944498 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.944499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.945710 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.945713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.946393 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.946394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.946471 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.946484 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.946550 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.946606 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":26,"thread_start_us":64,"threads_started":1}
I20260809 08:47:56.946728 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36190 (local address 127.10.184.67:34783)
0809 08:47:56.942728 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.942733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.942743 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:56.942780 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.942782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.942797 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.942876 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:56.944083 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944878 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.945897 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.946385 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.946453 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.946524 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.946593 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.946631 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260809 08:47:56.946993 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.10.184.66:33061)
0809 08:47:56.942994 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.942998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.943007 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:56.943042 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.943044 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.943059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.943313 (+ 254us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:56.943376 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944409 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.945796 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.946311 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.946368 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.946419 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.946881 (+ 462us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.946921 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":78,"threads_started":1}
I20260809 08:47:56.946695 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34783 (local address 127.0.0.1:36190)
0809 08:47:56.942534 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.942544 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.942597 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.942902 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.942906 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.942908 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:56.942969 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.942971 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944959 (+ 1988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.944960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.945815 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.945817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.946466 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.946467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.946527 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.946531 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.946581 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.946622 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":59,"threads_started":1}
I20260809 08:47:56.947682 23076 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (DEFAULT_TABLE table=test-table [id=59935ba569184097b6a5eb73236a7254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:56.947675 23302 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (DEFAULT_TABLE table=test-table [id=59935ba569184097b6a5eb73236a7254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:56.948035 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:56.948035 23302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:56.951638 23444 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Bootstrap starting.
I20260809 08:47:56.951884 23434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37321 (local address 127.0.0.1:50886)
0809 08:47:56.942573 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:56.942581 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:56.942620 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:56.943201 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:56.943202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:56.943205 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:56.943255 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.943257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944348 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.944349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.945710 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:56.945713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.951650 (+ 5937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.951651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:56.951714 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.951719 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:56.951779 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:47:56.951819 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":28,"spinlock_wait_cycles":384,"thread_start_us":52,"threads_started":1}
I20260809 08:47:56.951939 23446 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Bootstrap starting.
I20260809 08:47:56.952131 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:56.942776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50886 (local address 127.10.184.65:37321)
0809 08:47:56.942955 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:56.942959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:56.942969 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:47:56.943073 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:56.943075 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:56.943091 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:56.943324 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:56.943381 (+ 57us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.944268 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.946455 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:56.947179 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:56.947270 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:56.947342 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:56.951992 (+ 4650us) server_negotiation.cc:299] Negotiation successful
0809 08:47:56.952037 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":86,"threads_started":1}
I20260809 08:47:56.952452 23444 tablet_bootstrap.cc:654] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:56.952761 23446 tablet_bootstrap.cc:654] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:56.952704 22853 tablet_service.cc:1467] Processing CreateTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (DEFAULT_TABLE table=test-table [id=59935ba569184097b6a5eb73236a7254]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:56.952792 23444 log.cc:824] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:56.953034 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:56.953114 23446 log.cc:824] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:56.953769 23444 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: No bootstrap required, opened a new log
I20260809 08:47:56.953845 23444 ts_tablet_manager.cc:1397] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:56.953922 23446 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: No bootstrap required, opened a new log
I20260809 08:47:56.953989 23446 ts_tablet_manager.cc:1397] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:47:56.955278 23444 raft_consensus.cc:348] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.955407 23444 raft_consensus.cc:374] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:56.955439 23444 raft_consensus.cc:729] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Initialized, Role: FOLLOWER
I20260809 08:47:56.955549 23444 consensus_queue.cc:260] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.955806 23444 ts_tablet_manager.cc:1428] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:47:56.955811 23401 heartbeater.cc:503] Master 127.10.184.126:36159 was elected leader, sending a full tablet report...
I20260809 08:47:56.955919 23446 raft_consensus.cc:348] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.956074 23446 raft_consensus.cc:374] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:56.956113 23446 raft_consensus.cc:729] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Initialized, Role: FOLLOWER
I20260809 08:47:56.956249 23446 consensus_queue.cc:260] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.956521 23446 ts_tablet_manager.cc:1428] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 08:47:56.956650 23183 heartbeater.cc:503] Master 127.10.184.126:36159 was elected leader, sending a full tablet report...
I20260809 08:47:56.956866 23454 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Bootstrap starting.
I20260809 08:47:56.957469 23454 tablet_bootstrap.cc:654] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:56.957734 23454 log.cc:824] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:47:56.958415 23454 tablet_bootstrap.cc:492] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: No bootstrap required, opened a new log
I20260809 08:47:56.958468 23454 ts_tablet_manager.cc:1397] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:47:56.959703 23454 raft_consensus.cc:348] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.959820 23454 raft_consensus.cc:374] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:56.959846 23454 raft_consensus.cc:729] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Initialized, Role: FOLLOWER
I20260809 08:47:56.959924 23454 consensus_queue.cc:260] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:56.960137 23454 ts_tablet_manager.cc:1428] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:47:56.960198 22956 heartbeater.cc:503] Master 127.10.184.126:36159 was elected leader, sending a full tablet report...
W20260809 08:47:57.001163 23185 tablet.cc:2369] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:47:57.090858 22957 tablet.cc:2369] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:47:57.150446 23403 tablet.cc:2369] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:47:57.228363 23451 raft_consensus.cc:482] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:57.228484 23451 raft_consensus.cc:504] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.228788 23451 leader_election.cc:288] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.232656 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.229071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:45001 (local address 127.10.184.65:37321)
0809 08:47:57.229116 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.229118 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.229206 (+ 88us) server_negotiation.cc:407] Connection header received
0809 08:47:57.229251 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.229252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.229259 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.229317 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.229491 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.230244 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.231681 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.232287 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.232348 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.232357 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.232540 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.232582 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:47:57.232656 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.229158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:55279 (local address 127.10.184.67:34783)
0809 08:47:57.229210 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.229212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.229219 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:57.229248 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.229248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.229258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.229317 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.229490 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.230238 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.231681 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.232283 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.232348 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.232357 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.232540 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.232589 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:47:57.232674 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.228984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34783 (local address 127.10.184.66:55279)
0809 08:47:57.229036 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.229051 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.229106 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.229326 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.229327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.229328 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.229430 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.229432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.230311 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.230311 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.231599 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.231603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.232367 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.232368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.232471 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.232477 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.232529 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.232576 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":66}
I20260809 08:47:57.232702 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.228985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37321 (local address 127.10.184.66:45001)
0809 08:47:57.229149 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.229158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.229210 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.229326 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.229327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.229328 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.229430 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.229432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.230311 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.230312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.231599 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.231603 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.232368 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.232368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.232471 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.232478 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.232529 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.232576 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":82,"thread_start_us":61,"threads_started":1}
I20260809 08:47:57.233027 22883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5c6392d69b4cc9b61adbb3d2284ce7" candidate_uuid: "9bdeb752d89541308553e5f5731f0734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059ce69a2989456c8bfcd4909ebffe9b" is_pre_election: true
I20260809 08:47:57.233174 22883 raft_consensus.cc:2454] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bdeb752d89541308553e5f5731f0734 in term 0.
I20260809 08:47:57.233137 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5c6392d69b4cc9b61adbb3d2284ce7" candidate_uuid: "9bdeb752d89541308553e5f5731f0734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a" is_pre_election: true
I20260809 08:47:57.233297 23335 raft_consensus.cc:2454] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9bdeb752d89541308553e5f5731f0734 in term 0.
I20260809 08:47:57.233803 22992 leader_election.cc:302] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:57.233934 23451 raft_consensus.cc:2781] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:57.233997 23451 raft_consensus.cc:482] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:57.234021 23451 raft_consensus.cc:3037] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.234709 23451 raft_consensus.cc:504] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.234829 23451 leader_election.cc:288] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [CANDIDATE]: Term 1 election: Requested vote from peers 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.234958 22883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5c6392d69b4cc9b61adbb3d2284ce7" candidate_uuid: "9bdeb752d89541308553e5f5731f0734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059ce69a2989456c8bfcd4909ebffe9b"
I20260809 08:47:57.234967 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1d5c6392d69b4cc9b61adbb3d2284ce7" candidate_uuid: "9bdeb752d89541308553e5f5731f0734" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a"
I20260809 08:47:57.235033 23335 raft_consensus.cc:3037] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.235036 22883 raft_consensus.cc:3037] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.235692 23335 raft_consensus.cc:2454] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bdeb752d89541308553e5f5731f0734 in term 1.
I20260809 08:47:57.235698 22883 raft_consensus.cc:2454] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9bdeb752d89541308553e5f5731f0734 in term 1.
I20260809 08:47:57.235836 22992 leader_election.cc:302] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:57.235968 23451 raft_consensus.cc:2781] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:57.236105 23451 raft_consensus.cc:686] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 1 LEADER]: Becoming Leader. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Running, Role: LEADER
I20260809 08:47:57.236197 23451 consensus_queue.cc:237] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.236994 22636 catalog_manager.cc:5495] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9bdeb752d89541308553e5f5731f0734 (127.10.184.66). New cstate: current_term: 1 leader_uuid: "9bdeb752d89541308553e5f5731f0734" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:57.295902 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.293835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.10.184.66:33061)
0809 08:47:57.293897 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.293899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.293906 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:57.293941 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.293942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.293958 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.294014 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.294113 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.294574 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.295302 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.295485 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.295530 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.295645 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.295744 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.295797 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.295816 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.295831 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:47:57.297577 23335 raft_consensus.cc:1261] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Refusing update from remote peer 9bdeb752d89541308553e5f5731f0734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.297577 22883 raft_consensus.cc:1261] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Refusing update from remote peer 9bdeb752d89541308553e5f5731f0734: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.297894 23463 consensus_queue.cc:1035] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.298024 23451 consensus_queue.cc:1035] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [LEADER]: Connected to new peer: Peer: permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.300191 23467 mvcc.cc:205] Tried to move back new op lower bound from 7316542575808229376 to 7316542575559954432. Current Snapshot: MvccSnapshot[applied={T|T < 7316542575808229376}]
I20260809 08:47:57.302289 22636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.302407 22636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.303155 22636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Sending DeleteTablet for 3 replicas of tablet 1d5c6392d69b4cc9b61adbb3d2284ce7
I20260809 08:47:57.303488 22853 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:50886
I20260809 08:47:57.303552 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:60448
I20260809 08:47:57.303719 23480 tablet_replica.cc:330] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: stopping tablet replica
I20260809 08:47:57.303642 22636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:57.303791 22636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:57.303944 23480 raft_consensus.cc:2229] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.304016 23302 tablet_service.cc:1514] Processing DeleteTablet for tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36190
I20260809 08:47:57.304196 23480 raft_consensus.cc:2258] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.304266 23481 tablet_replica.cc:330] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: stopping tablet replica
I20260809 08:47:57.304419 23481 raft_consensus.cc:2229] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:57.304575 23481 raft_consensus.cc:2258] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.304687 23480 ts_tablet_manager.cc:1905] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.305105 23481 ts_tablet_manager.cc:1905] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.305277 23076 tablet_service.cc:1467] Processing CreateTablet for tablet d13082d768f74287978d52ac2dd6b740 (DEFAULT_TABLE table=test-table [id=ee440869df414866976a8376dbb8b185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.305392 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13082d768f74287978d52ac2dd6b740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.306449 23480 ts_tablet_manager.cc:1918] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.306542 23480 log.cc:1196] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/1d5c6392d69b4cc9b61adbb3d2284ce7
I20260809 08:47:57.306773 23446 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Bootstrap starting.
I20260809 08:47:57.306840 23302 tablet_service.cc:1467] Processing CreateTablet for tablet d13082d768f74287978d52ac2dd6b740 (DEFAULT_TABLE table=test-table [id=ee440869df414866976a8376dbb8b185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.306861 23481 ts_tablet_manager.cc:1918] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.306938 23481 log.cc:1196] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/1d5c6392d69b4cc9b61adbb3d2284ce7
I20260809 08:47:57.306942 23302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13082d768f74287978d52ac2dd6b740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.307164 23446 tablet_bootstrap.cc:654] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.307207 23481 ts_tablet_manager.cc:1939] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 9bdeb752d89541308553e5f5731f0734: Deleting consensus metadata
I20260809 08:47:57.307472 23480 ts_tablet_manager.cc:1939] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting consensus metadata
I20260809 08:47:57.308161 23446 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: No bootstrap required, opened a new log
I20260809 08:47:57.308203 23446 ts_tablet_manager.cc:1397] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:57.308336 23446 raft_consensus.cc:348] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.308440 23446 raft_consensus.cc:374] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.308465 23446 raft_consensus.cc:729] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.308506 23446 consensus_queue.cc:260] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.308518 22608 catalog_manager.cc:4841] TS 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061): tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (table test-table [id=59935ba569184097b6a5eb73236a7254]) successfully deleted
I20260809 08:47:57.308595 23446 ts_tablet_manager.cc:1428] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:57.308666 22607 catalog_manager.cc:4841] TS 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321): tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (table test-table [id=59935ba569184097b6a5eb73236a7254]) successfully deleted
I20260809 08:47:57.308687 23444 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Bootstrap starting.
I20260809 08:47:57.308826 22853 tablet_service.cc:1467] Processing CreateTablet for tablet d13082d768f74287978d52ac2dd6b740 (DEFAULT_TABLE table=test-table [id=ee440869df414866976a8376dbb8b185]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.309026 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d13082d768f74287978d52ac2dd6b740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.309075 23484 tablet_replica.cc:330] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: stopping tablet replica
I20260809 08:47:57.309161 23484 raft_consensus.cc:2229] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.309216 23444 tablet_bootstrap.cc:654] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.309260 23484 raft_consensus.cc:2258] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.310014 23454 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Bootstrap starting.
I20260809 08:47:57.310410 23454 tablet_bootstrap.cc:654] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.312039 23444 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: No bootstrap required, opened a new log
I20260809 08:47:57.312088 23454 tablet_bootstrap.cc:492] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: No bootstrap required, opened a new log
I20260809 08:47:57.312095 23444 ts_tablet_manager.cc:1397] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260809 08:47:57.312131 23454 ts_tablet_manager.cc:1397] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:47:57.312242 23444 raft_consensus.cc:348] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.312263 23454 raft_consensus.cc:348] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.312309 23444 raft_consensus.cc:374] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.312330 23454 raft_consensus.cc:374] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.312332 23444 raft_consensus.cc:729] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.312350 23454 raft_consensus.cc:729] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.312381 23444 consensus_queue.cc:260] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.312388 23454 consensus_queue.cc:260] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.312479 23444 ts_tablet_manager.cc:1428] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.312497 23454 ts_tablet_manager.cc:1428] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.312564 23484 ts_tablet_manager.cc:1905] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.313908 23484 ts_tablet_manager.cc:1918] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.313966 23484 log.cc:1196] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/1d5c6392d69b4cc9b61adbb3d2284ce7
I20260809 08:47:57.314160 23484 ts_tablet_manager.cc:1939] T 1d5c6392d69b4cc9b61adbb3d2284ce7 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting consensus metadata
I20260809 08:47:57.314497 22613 catalog_manager.cc:4841] TS 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783): tablet 1d5c6392d69b4cc9b61adbb3d2284ce7 (table test-table [id=59935ba569184097b6a5eb73236a7254]) successfully deleted
I20260809 08:47:57.590389 23457 raft_consensus.cc:482] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:57.590485 23457 raft_consensus.cc:504] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.590797 23457 leader_election.cc:288] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.594296 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.591035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:42175 (local address 127.10.184.67:34783)
0809 08:47:57.591082 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.591084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.591158 (+ 74us) server_negotiation.cc:407] Connection header received
0809 08:47:57.591266 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.591267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.591278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.591306 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.591522 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.592060 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.593548 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.593932 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.593993 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.594001 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.594181 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.594231 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:57.594321 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.591190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:55327 (local address 127.10.184.66:33061)
0809 08:47:57.591232 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.591234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.591242 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:57.591305 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.591305 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.591315 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.591363 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.591514 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.592239 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.593569 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.593973 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.594038 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.594046 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.594200 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.594243 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:57.594316 23492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.590971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34783 (local address 127.10.184.65:42175)
0809 08:47:57.591090 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.591099 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.591189 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.591375 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.591376 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.591378 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.591434 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.591435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.592128 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.592129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.593457 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.593460 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.594024 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.594026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.594100 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.594105 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.594154 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.594195 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":4,"thread_start_us":58,"threads_started":1}
I20260809 08:47:57.594326 23493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.591014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:33061 (local address 127.10.184.65:55327)
0809 08:47:57.591192 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.591202 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.591249 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.591403 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.591403 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.591405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.591432 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.591435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.592317 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.592318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.593458 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.593460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.594078 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.594080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.594129 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.594133 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.594188 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.594229 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":2,"thread_start_us":82,"threads_started":1}
I20260809 08:47:57.594544 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13082d768f74287978d52ac2dd6b740" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a" is_pre_election: true
I20260809 08:47:57.594640 23335 raft_consensus.cc:2454] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:57.594816 22784 leader_election.cc:302] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 1ed79483d3344acfb4a3e0c6693d015a; no voters:
I20260809 08:47:57.594853 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13082d768f74287978d52ac2dd6b740" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734" is_pre_election: true
I20260809 08:47:57.594945 23457 raft_consensus.cc:2781] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:57.594982 23457 raft_consensus.cc:482] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:57.594981 23110 raft_consensus.cc:2454] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:57.595011 23457 raft_consensus.cc:3037] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.595745 23457 raft_consensus.cc:504] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.595862 23457 leader_election.cc:288] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Requested vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.595992 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13082d768f74287978d52ac2dd6b740" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734"
I20260809 08:47:57.596021 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d13082d768f74287978d52ac2dd6b740" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a"
I20260809 08:47:57.596074 23110 raft_consensus.cc:3037] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.596081 23335 raft_consensus.cc:3037] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.596762 23335 raft_consensus.cc:2454] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:57.596850 23110 raft_consensus.cc:2454] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:57.596946 22784 leader_election.cc:302] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 1ed79483d3344acfb4a3e0c6693d015a; no voters:
I20260809 08:47:57.597064 23457 raft_consensus.cc:2781] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:57.597220 23457 raft_consensus.cc:686] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Becoming Leader. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Running, Role: LEADER
I20260809 08:47:57.597299 23457 consensus_queue.cc:237] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.597944 22636 catalog_manager.cc:5495] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65). New cstate: current_term: 1 leader_uuid: "059ce69a2989456c8bfcd4909ebffe9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } health_report { overall_health: UNKNOWN } } }
Aug 09 08:47:57 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265276, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.126@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:57.691794 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.685516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:36159 (local address 127.0.0.1:38784)
0809 08:47:57.685703 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.685713 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.685770 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.685922 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.685922 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.686229 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:47:57.686283 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.686285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.686890 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.686890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.687418 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.687419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.687732 (+ 313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.687734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.687787 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.688273 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:47:57.688278 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:47:57.689611 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:47:57.690927 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:57.690934 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:57.690935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:57.691128 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:57.691288 (+ 160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:47:57.691288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:47:57.691288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:47:57.691340 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:47:57.691505 (+ 165us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:47:57.691510 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:47:57.691548 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.691592 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.691703 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":2,"thread_start_us":83,"threads_started":1}
I20260809 08:47:57.691854 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.685702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38784 (local address 127.10.184.126:36159)
0809 08:47:57.685814 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.685815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.685823 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:47:57.685842 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.685843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.685851 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.685910 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:57.686340 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.686839 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.687471 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.687653 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.687703 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.689704 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:57.689706 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:57.689706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:57.689712 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:57.690865 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:57.691198 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:57.691199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:57.691199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:57.691246 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:57.691377 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:57.691377 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:57.691377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:57.691467 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:57.691511 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.691613 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.691750 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":91,"thread_start_us":60,"threads_started":1}
I20260809 08:47:57.693704 22636 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:38784:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 08:47:57.693905 22636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:57.695256 22853 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5cc1d3ec5045439e3b41f7a427340d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13bdadbcf0f644d9aa2245392df6e4b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:57.695408 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5cc1d3ec5045439e3b41f7a427340d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.696666 23454 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: Bootstrap starting.
I20260809 08:47:57.696762 23076 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5cc1d3ec5045439e3b41f7a427340d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13bdadbcf0f644d9aa2245392df6e4b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:57.697036 23302 tablet_service.cc:1467] Processing CreateTablet for tablet 1a5cc1d3ec5045439e3b41f7a427340d (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=13bdadbcf0f644d9aa2245392df6e4b1]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 08:47:57.697168 23302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5cc1d3ec5045439e3b41f7a427340d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.697191 23454 tablet_bootstrap.cc:654] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.697834 23454 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: No bootstrap required, opened a new log
I20260809 08:47:57.697877 23454 ts_tablet_manager.cc:1397] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:57.698009 23454 raft_consensus.cc:348] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.698067 23454 raft_consensus.cc:374] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.698086 23454 raft_consensus.cc:729] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.698127 23454 consensus_queue.cc:260] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.698233 23454 ts_tablet_manager.cc:1428] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.698298 23457 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.698397 23457 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.698580 23444 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: Bootstrap starting.
I20260809 08:47:57.696883 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1a5cc1d3ec5045439e3b41f7a427340d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.699015 23444 tablet_bootstrap.cc:654] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.699659 23444 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: No bootstrap required, opened a new log
I20260809 08:47:57.699707 23444 ts_tablet_manager.cc:1397] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:47:57.699836 23444 raft_consensus.cc:348] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.699888 23444 raft_consensus.cc:374] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.699906 23444 raft_consensus.cc:729] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.699954 23444 consensus_queue.cc:260] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.700047 23444 ts_tablet_manager.cc:1428] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.700188 23450 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.700291 23450 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.700373 23446 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: Bootstrap starting.
I20260809 08:47:57.700778 23446 tablet_bootstrap.cc:654] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.701370 23446 tablet_bootstrap.cc:492] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: No bootstrap required, opened a new log
I20260809 08:47:57.701418 23446 ts_tablet_manager.cc:1397] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:57.701514 23446 raft_consensus.cc:348] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.701565 23446 raft_consensus.cc:374] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.701583 23446 raft_consensus.cc:729] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.701627 23446 consensus_queue.cc:260] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.701719 23446 ts_tablet_manager.cc:1428] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.701810 23463 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.701884 23463 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.750202 23450 raft_consensus.cc:482] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:57.750298 23450 raft_consensus.cc:504] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.750617 23450 leader_election.cc:288] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321), 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061)
I20260809 08:47:57.753913 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.750786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37321 (local address 127.10.184.67:49915)
0809 08:47:57.750952 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.750962 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.751010 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.751274 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.751275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.751276 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.751327 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.751328 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.752128 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.752130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.753113 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.753115 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.753704 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.753705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.753772 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.753777 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.753817 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.753845 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":44,"thread_start_us":57,"threads_started":1}
I20260809 08:47:57.753968 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.751061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:49915 (local address 127.10.184.65:37321)
0809 08:47:57.751159 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.751161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.751168 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:57.751190 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.751190 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.751198 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.751250 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.751381 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.752068 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.753192 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.753619 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.753696 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.753705 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.753834 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.753873 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":55,"threads_started":1}
I20260809 08:47:57.754307 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.750957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59879 (local address 127.10.184.66:33061)
0809 08:47:57.750997 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.750999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.751006 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:57.751032 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.751033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.751041 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.751091 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:47:57.751254 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.752012 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.753480 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.753957 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.754023 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.754032 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.754216 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.754259 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:57.754318 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.750780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:33061 (local address 127.10.184.67:59879)
0809 08:47:57.750889 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.750902 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.750959 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.751097 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.751098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.751100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:47:57.751174 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.751175 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.752088 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.752089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.753394 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.753397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.754068 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.754069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.754142 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.754146 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.754204 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.754242 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":34,"thread_start_us":55,"threads_started":1}
I20260809 08:47:57.754311 22883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5cc1d3ec5045439e3b41f7a427340d" candidate_uuid: "1ed79483d3344acfb4a3e0c6693d015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059ce69a2989456c8bfcd4909ebffe9b" is_pre_election: true
I20260809 08:47:57.754401 22883 raft_consensus.cc:2454] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ed79483d3344acfb4a3e0c6693d015a in term 0.
I20260809 08:47:57.754580 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5cc1d3ec5045439e3b41f7a427340d" candidate_uuid: "1ed79483d3344acfb4a3e0c6693d015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734" is_pre_election: true
I20260809 08:47:57.754606 23220 leader_election.cc:302] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 1ed79483d3344acfb4a3e0c6693d015a; no voters:
I20260809 08:47:57.754657 23110 raft_consensus.cc:2454] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1ed79483d3344acfb4a3e0c6693d015a in term 0.
I20260809 08:47:57.754745 23450 raft_consensus.cc:2781] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:57.754787 23450 raft_consensus.cc:482] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:57.754805 23450 raft_consensus.cc:3037] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.755446 23450 raft_consensus.cc:504] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.755568 23450 leader_election.cc:288] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [CANDIDATE]: Term 1 election: Requested vote from peers 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321), 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061)
I20260809 08:47:57.755705 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5cc1d3ec5045439e3b41f7a427340d" candidate_uuid: "1ed79483d3344acfb4a3e0c6693d015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734"
I20260809 08:47:57.755724 22883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1a5cc1d3ec5045439e3b41f7a427340d" candidate_uuid: "1ed79483d3344acfb4a3e0c6693d015a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "059ce69a2989456c8bfcd4909ebffe9b"
I20260809 08:47:57.755769 23110 raft_consensus.cc:3037] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.755790 22883 raft_consensus.cc:3037] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.756341 23110 raft_consensus.cc:2454] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ed79483d3344acfb4a3e0c6693d015a in term 1.
I20260809 08:47:57.756368 22883 raft_consensus.cc:2454] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1ed79483d3344acfb4a3e0c6693d015a in term 1.
I20260809 08:47:57.756526 23222 leader_election.cc:302] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ed79483d3344acfb4a3e0c6693d015a, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:57.756614 23450 raft_consensus.cc:2781] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:57.756714 23450 raft_consensus.cc:686] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [term 1 LEADER]: Becoming Leader. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Running, Role: LEADER
I20260809 08:47:57.756778 23450 consensus_queue.cc:237] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.757112 23514 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: TxnStatusTablet state changed. Reason: New leader 1ed79483d3344acfb4a3e0c6693d015a. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.757190 23514 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:57.757457 23516 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 08:47:57.757459 22636 catalog_manager.cc:5495] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67). New cstate: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } health_report { overall_health: HEALTHY } } }
I20260809 08:47:57.757560 23516 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 08:47:57.771909 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.769635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36206 (local address 127.10.184.67:34783)
0809 08:47:57.769686 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.769687 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.769696 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:57.769778 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.769778 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.769795 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.769848 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.769963 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.770487 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.771197 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.771398 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.771450 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.771611 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.771708 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.771767 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.771814 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.771832 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:47:57.771926 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.769590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34783 (local address 127.0.0.1:36206)
0809 08:47:57.769663 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.769672 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.769717 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.769851 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.769851 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.769852 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:57.769907 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.769910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.770563 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.770565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.771127 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.771129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.771456 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.771457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.771519 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.771536 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.771762 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.771770 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.771812 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.771863 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:47:57.772976 22883 raft_consensus.cc:1261] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Refusing update from remote peer 1ed79483d3344acfb4a3e0c6693d015a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.773092 23110 raft_consensus.cc:1261] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Refusing update from remote peer 1ed79483d3344acfb4a3e0c6693d015a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.773252 23514 consensus_queue.cc:1035] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [LEADER]: Connected to new peer: Peer: permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.773360 23450 consensus_queue.cc:1035] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.773958 23457 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: TxnStatusTablet state changed. Reason: New leader 1ed79483d3344acfb4a3e0c6693d015a. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.773975 23463 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: TxnStatusTablet state changed. Reason: New leader 1ed79483d3344acfb4a3e0c6693d015a. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.774041 23457 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.774055 23463 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.774727 23450 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.774804 23450 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:57.774873 23470 mvcc.cc:205] Tried to move back new op lower bound from 7316542577756258304 to 7316542577692303360. Current Snapshot: MvccSnapshot[applied={T|T < 7316542577756258304 or (T in {7316542577756258304})}]
I20260809 08:47:57.775274 23463 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.775359 23463 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 9bdeb752d89541308553e5f5731f0734: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.775698 23450 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.775774 23450 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 1ed79483d3344acfb4a3e0c6693d015a: This TxnStatusTablet replica's current role is: LEADER
I20260809 08:47:57.775933 23469 mvcc.cc:205] Tried to move back new op lower bound from 7316542577756258304 to 7316542577692303360. Current Snapshot: MvccSnapshot[applied={T|T < 7316542577756258304}]
I20260809 08:47:57.776132 23457 tablet_replica.cc:439] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1ed79483d3344acfb4a3e0c6693d015a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } } }
I20260809 08:47:57.776216 23457 tablet_replica.cc:442] T 1a5cc1d3ec5045439e3b41f7a427340d P 059ce69a2989456c8bfcd4909ebffe9b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 08:47:57.778616 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.776492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.10.184.65:37321)
0809 08:47:57.776544 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.776546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.776553 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:47:57.776586 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.776587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.776605 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.776680 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.776776 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.777280 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.777954 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.778150 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.778204 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.778346 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.778440 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.778498 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.778538 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.778558 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:47:57.783874 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.780936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:54475 (local address 127.10.184.67:34783)
0809 08:47:57.780977 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.780979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.781121 (+ 142us) server_negotiation.cc:407] Connection header received
0809 08:47:57.781188 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.781189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.781196 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.781238 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.781363 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.781784 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.783349 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.783515 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.783557 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.783718 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.783740 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.783769 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.783819 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.783831 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:47:57.783951 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.780857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34783 (local address 127.10.184.65:54475)
0809 08:47:57.781054 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.781064 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.781125 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.781242 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.781243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.781244 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:57.781302 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.781304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.781843 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.781844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.783280 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.783283 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.783570 (+ 287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.783571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.783637 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.783652 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.783774 (+ 122us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.783775 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.783817 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.783860 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
I20260809 08:47:57.785905 23335 raft_consensus.cc:1261] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.785900 23110 raft_consensus.cc:1261] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.786150 23457 consensus_queue.cc:1035] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.786280 23494 consensus_queue.cc:1035] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.794925 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.792152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:38477 (local address 127.10.184.65:37321)
0809 08:47:57.792203 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.792205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.792309 (+ 104us) server_negotiation.cc:407] Connection header received
0809 08:47:57.792382 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.792382 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.792391 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.792441 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.792551 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.793022 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.794273 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.794493 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.794550 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.794741 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.794765 (+ 24us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.794798 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.794854 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.794874 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:47:57.795128 23537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.792097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37321 (local address 127.10.184.67:38477)
0809 08:47:57.792256 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:57.792267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:47:57.792321 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:47:57.792444 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:47:57.792445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:47:57.792446 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 08:47:57.792499 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.792501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.793082 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.793083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.794200 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:47:57.794204 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.794567 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.794568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:47:57.794630 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.794644 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.794811 (+ 167us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.794813 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:47:57.794853 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:47:57.794893 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260809 08:47:57.803853 22636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.803943 22636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.807711 22636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Sending DeleteTablet for 3 replicas of tablet d13082d768f74287978d52ac2dd6b740
I20260809 08:47:57.807981 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet d13082d768f74287978d52ac2dd6b740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:60448
I20260809 08:47:57.807999 23302 tablet_service.cc:1514] Processing DeleteTablet for tablet d13082d768f74287978d52ac2dd6b740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36190
I20260809 08:47:57.808194 23484 tablet_replica.cc:330] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: stopping tablet replica
I20260809 08:47:57.808264 23484 raft_consensus.cc:2229] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.808295 23484 raft_consensus.cc:2258] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.808313 22853 tablet_service.cc:1514] Processing DeleteTablet for tablet d13082d768f74287978d52ac2dd6b740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:50886
I20260809 08:47:57.808598 23484 ts_tablet_manager.cc:1905] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.810611 23484 ts_tablet_manager.cc:1918] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:57.810670 23484 log.cc:1196] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/d13082d768f74287978d52ac2dd6b740
I20260809 08:47:57.810873 23484 ts_tablet_manager.cc:1939] T d13082d768f74287978d52ac2dd6b740 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting consensus metadata
I20260809 08:47:57.811360 22613 catalog_manager.cc:4841] TS 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783): tablet d13082d768f74287978d52ac2dd6b740 (table test-table [id=ee440869df414866976a8376dbb8b185]) successfully deleted
I20260809 08:47:57.813129 23542 tablet_replica.cc:330] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: stopping tablet replica
I20260809 08:47:57.813202 23542 raft_consensus.cc:2229] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.813242 23542 raft_consensus.cc:2258] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.812991 23543 tablet_replica.cc:330] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: stopping tablet replica
I20260809 08:47:57.813341 23543 raft_consensus.cc:2229] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:57.813431 23543 raft_consensus.cc:2258] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.813639 23543 ts_tablet_manager.cc:1905] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.814507 23542 ts_tablet_manager.cc:1905] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.815254 23543 ts_tablet_manager.cc:1918] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:57.815315 23543 log.cc:1196] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/d13082d768f74287978d52ac2dd6b740
I20260809 08:47:57.815521 23543 ts_tablet_manager.cc:1939] T d13082d768f74287978d52ac2dd6b740 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting consensus metadata
I20260809 08:47:57.815807 22607 catalog_manager.cc:4841] TS 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321): tablet d13082d768f74287978d52ac2dd6b740 (table test-table [id=ee440869df414866976a8376dbb8b185]) successfully deleted
I20260809 08:47:57.816067 23542 ts_tablet_manager.cc:1918] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:57.816123 23542 log.cc:1196] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/d13082d768f74287978d52ac2dd6b740
I20260809 08:47:57.816316 23542 ts_tablet_manager.cc:1939] T d13082d768f74287978d52ac2dd6b740 P 9bdeb752d89541308553e5f5731f0734: Deleting consensus metadata
I20260809 08:47:57.816656 22608 catalog_manager.cc:4841] TS 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061): tablet d13082d768f74287978d52ac2dd6b740 (table test-table [id=ee440869df414866976a8376dbb8b185]) successfully deleted
Aug 09 08:47:57 dist-test-slave-ncp9 krb5kdc[22567](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 08:47:57 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:47:57 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:47:57.842545 23503 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:47:57.835711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38794 (local address 127.10.184.126:36159)
0809 08:47:57.835769 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.835772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.835902 (+ 130us) server_negotiation.cc:407] Connection header received
0809 08:47:57.835966 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.835967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.835979 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.836042 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:57.836759 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.837622 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.838752 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.839063 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.839137 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.842477 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38794: BlockingRecv error: recv got EOF from 127.0.0.1:38794 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 08:47:57 dist-test-slave-ncp9 krb5kdc[22567](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM
I20260809 08:47:57.851936 23503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.843344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.10.184.126:36159)
0809 08:47:57.843380 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.843382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.843643 (+ 261us) server_negotiation.cc:407] Connection header received
0809 08:47:57.843693 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.843694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.843703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.843761 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:47:57.844323 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.845124 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.846125 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.846436 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.846502 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.849161 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:47:57.849163 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:47:57.849163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:47:57.849171 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:47:57.850764 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:57.851161 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:57.851162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:57.851163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:57.851226 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:47:57.851422 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:47:57.851422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:47:57.851423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:47:57.851553 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:47:57.851656 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.851763 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.851842 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:47:57.853524 22636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:57.853698 22636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:57.855037 22853 tablet_service.cc:1467] Processing CreateTablet for tablet 6a902c7087014cfe9747f7f1c612d110 (DEFAULT_TABLE table=test-table [id=98d83af26f1746d697950e52c73087cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.855187 22853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a902c7087014cfe9747f7f1c612d110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.856578 23454 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Bootstrap starting.
I20260809 08:47:57.856616 23076 tablet_service.cc:1467] Processing CreateTablet for tablet 6a902c7087014cfe9747f7f1c612d110 (DEFAULT_TABLE table=test-table [id=98d83af26f1746d697950e52c73087cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.856742 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a902c7087014cfe9747f7f1c612d110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.857136 23454 tablet_bootstrap.cc:654] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.858201 23446 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Bootstrap starting.
I20260809 08:47:57.858529 23446 tablet_bootstrap.cc:654] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.858789 23302 tablet_service.cc:1467] Processing CreateTablet for tablet 6a902c7087014cfe9747f7f1c612d110 (DEFAULT_TABLE table=test-table [id=98d83af26f1746d697950e52c73087cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.858897 23302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a902c7087014cfe9747f7f1c612d110. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.860209 23444 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Bootstrap starting.
I20260809 08:47:57.860623 23446 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: No bootstrap required, opened a new log
I20260809 08:47:57.860674 23446 ts_tablet_manager.cc:1397] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:57.860728 23444 tablet_bootstrap.cc:654] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.860776 23446 raft_consensus.cc:348] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.860838 23446 raft_consensus.cc:374] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.860857 23446 raft_consensus.cc:729] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.860921 23446 consensus_queue.cc:260] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.861011 23446 ts_tablet_manager.cc:1428] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.861543 23444 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: No bootstrap required, opened a new log
I20260809 08:47:57.861584 23444 ts_tablet_manager.cc:1397] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:57.861732 23444 raft_consensus.cc:348] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.861795 23444 raft_consensus.cc:374] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.861814 23444 raft_consensus.cc:729] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.861861 23444 consensus_queue.cc:260] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.861953 23444 ts_tablet_manager.cc:1428] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.862489 23454 tablet_bootstrap.cc:492] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: No bootstrap required, opened a new log
I20260809 08:47:57.862542 23454 ts_tablet_manager.cc:1397] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260809 08:47:57.862684 23454 raft_consensus.cc:348] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.862748 23454 raft_consensus.cc:374] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.862766 23454 raft_consensus.cc:729] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.862811 23454 consensus_queue.cc:260] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.862892 23454 ts_tablet_manager.cc:1428] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.896190 23538 raft_consensus.cc:482] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:57.896320 23538 raft_consensus.cc:504] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.896489 23538 leader_election.cc:288] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.896724 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a902c7087014cfe9747f7f1c612d110" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734" is_pre_election: true
I20260809 08:47:57.896751 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a902c7087014cfe9747f7f1c612d110" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a" is_pre_election: true
I20260809 08:47:57.896823 23110 raft_consensus.cc:2454] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:57.896859 23335 raft_consensus.cc:2454] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:57.897017 22782 leader_election.cc:302] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:57.897126 23538 raft_consensus.cc:2781] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:57.897158 23538 raft_consensus.cc:482] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:57.897176 23538 raft_consensus.cc:3037] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.898108 23538 raft_consensus.cc:504] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.898247 23538 leader_election.cc:288] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Requested vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:57.898386 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a902c7087014cfe9747f7f1c612d110" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a"
I20260809 08:47:57.898468 23335 raft_consensus.cc:3037] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.898435 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a902c7087014cfe9747f7f1c612d110" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734"
I20260809 08:47:57.898504 23110 raft_consensus.cc:3037] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:57.899305 23110 raft_consensus.cc:2454] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:57.899379 23335 raft_consensus.cc:2454] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:57.899510 22782 leader_election.cc:302] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:57.899577 23538 raft_consensus.cc:2781] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:57.899611 23538 raft_consensus.cc:686] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Becoming Leader. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Running, Role: LEADER
I20260809 08:47:57.899654 23538 consensus_queue.cc:237] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.900795 22636 catalog_manager.cc:5495] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65). New cstate: current_term: 1 leader_uuid: "059ce69a2989456c8bfcd4909ebffe9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:57.910105 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:57.907690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50904 (local address 127.10.184.65:37321)
0809 08:47:57.907731 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:47:57.907733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:47:57.907742 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:47:57.907803 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:47:57.907804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:47:57.907812 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:47:57.907855 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:47:57.907981 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.908451 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.909519 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:47:57.909702 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:47:57.909751 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:47:57.909928 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:47:57.909954 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:47:57.909986 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 08:47:57.910036 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 08:47:57.910050 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:47:57.910786 23335 raft_consensus.cc:1261] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.910795 23110 raft_consensus.cc:1261] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:57.911007 23494 consensus_queue.cc:1035] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.911124 23538 consensus_queue.cc:1035] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:57.913503 22636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.913599 22636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:57.914436 22636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Sending DeleteTablet for 3 replicas of tablet 6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:57.914716 22853 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a902c7087014cfe9747f7f1c612d110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:50886
I20260809 08:47:57.914726 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a902c7087014cfe9747f7f1c612d110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:60448
I20260809 08:47:57.914832 22852 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a902c7087014cfe9747f7f1c612d110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:50886
I20260809 08:47:57.914906 23543 tablet_replica.cc:330] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: stopping tablet replica
I20260809 08:47:57.914955 23543 raft_consensus.cc:2229] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:57.915009 23543 raft_consensus.cc:2258] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.915081 23542 tablet_replica.cc:330] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: stopping tablet replica
I20260809 08:47:57.915135 23542 raft_consensus.cc:2229] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.915177 23542 raft_consensus.cc:2258] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.915426 23543 ts_tablet_manager.cc:1905] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.915611 23542 ts_tablet_manager.cc:1905] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.916842 22636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:47:57.916996 22636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:47:57.917049 23543 ts_tablet_manager.cc:1918] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.917122 23543 log.cc:1196] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:57.917318 23543 ts_tablet_manager.cc:1939] T 6a902c7087014cfe9747f7f1c612d110 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting consensus metadata
I20260809 08:47:57.917625 22607 catalog_manager.cc:4841] TS 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321): tablet 6a902c7087014cfe9747f7f1c612d110 (table test-table [id=98d83af26f1746d697950e52c73087cb]) successfully deleted
W20260809 08:47:57.917750 22607 catalog_manager.cc:4808] TS 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321): delete failed for tablet 6a902c7087014cfe9747f7f1c612d110 because the tablet was not found. No further retry: Not found: Tablet not found: 6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:57.917809 23302 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a902c7087014cfe9747f7f1c612d110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36190
I20260809 08:47:57.917924 23301 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a902c7087014cfe9747f7f1c612d110 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:57 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36190
I20260809 08:47:57.917944 23484 tablet_replica.cc:330] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: stopping tablet replica
I20260809 08:47:57.918011 23484 raft_consensus.cc:2229] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:57.918046 23484 raft_consensus.cc:2258] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:57.918558 23542 ts_tablet_manager.cc:1918] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.918618 23542 log.cc:1196] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:57.918820 23542 ts_tablet_manager.cc:1939] T 6a902c7087014cfe9747f7f1c612d110 P 9bdeb752d89541308553e5f5731f0734: Deleting consensus metadata
I20260809 08:47:57.919119 22608 catalog_manager.cc:4841] TS 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061): tablet 6a902c7087014cfe9747f7f1c612d110 (table test-table [id=98d83af26f1746d697950e52c73087cb]) successfully deleted
I20260809 08:47:57.919432 23076 tablet_service.cc:1467] Processing CreateTablet for tablet 428c6b3f791b462aade58d7af08053e4 (DEFAULT_TABLE table=test-table [id=30462ab5765c44e1ba04ff93d2b6f593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.919423 22852 tablet_service.cc:1467] Processing CreateTablet for tablet 428c6b3f791b462aade58d7af08053e4 (DEFAULT_TABLE table=test-table [id=30462ab5765c44e1ba04ff93d2b6f593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.919479 23301 tablet_service.cc:1467] Processing CreateTablet for tablet 428c6b3f791b462aade58d7af08053e4 (DEFAULT_TABLE table=test-table [id=30462ab5765c44e1ba04ff93d2b6f593]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:47:57.919529 23076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428c6b3f791b462aade58d7af08053e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.919533 22852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428c6b3f791b462aade58d7af08053e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.919612 23301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 428c6b3f791b462aade58d7af08053e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:47:57.921049 23444 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Bootstrap starting.
I20260809 08:47:57.921070 23446 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Bootstrap starting.
I20260809 08:47:57.921381 23444 tablet_bootstrap.cc:654] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.921399 23446 tablet_bootstrap.cc:654] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.921641 23454 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: Bootstrap starting.
I20260809 08:47:57.921873 23444 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: No bootstrap required, opened a new log
I20260809 08:47:57.921905 23444 ts_tablet_manager.cc:1397] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 08:47:57.921986 23444 raft_consensus.cc:348] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.922030 23444 raft_consensus.cc:374] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.922042 23444 raft_consensus.cc:729] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ed79483d3344acfb4a3e0c6693d015a, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.922036 23454 tablet_bootstrap.cc:654] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: Neither blocks nor log segments found. Creating new log.
I20260809 08:47:57.922063 23444 consensus_queue.cc:260] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.922125 23444 ts_tablet_manager.cc:1428] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.922982 23446 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: No bootstrap required, opened a new log
I20260809 08:47:57.923031 23446 ts_tablet_manager.cc:1397] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:47:57.923148 23446 raft_consensus.cc:348] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.923213 23446 raft_consensus.cc:374] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.923229 23446 raft_consensus.cc:729] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bdeb752d89541308553e5f5731f0734, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.923271 23446 consensus_queue.cc:260] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.923349 23446 ts_tablet_manager.cc:1428] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 08:47:57.923625 23454 tablet_bootstrap.cc:492] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: No bootstrap required, opened a new log
I20260809 08:47:57.923674 23454 ts_tablet_manager.cc:1397] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 08:47:57.923804 23454 raft_consensus.cc:348] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.923867 23454 raft_consensus.cc:374] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:47:57.923893 23454 raft_consensus.cc:729] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Initialized, Role: FOLLOWER
I20260809 08:47:57.923933 23454 consensus_queue.cc:260] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:57.924011 23454 ts_tablet_manager.cc:1428] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 08:47:57.924327 23484 ts_tablet_manager.cc:1905] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:57.925464 23484 ts_tablet_manager.cc:1918] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:47:57.925511 23484 log.cc:1196] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:57.925683 23484 ts_tablet_manager.cc:1939] T 6a902c7087014cfe9747f7f1c612d110 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting consensus metadata
I20260809 08:47:57.926010 22613 catalog_manager.cc:4841] TS 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783): tablet 6a902c7087014cfe9747f7f1c612d110 (table test-table [id=98d83af26f1746d697950e52c73087cb]) successfully deleted
W20260809 08:47:57.926085 22613 catalog_manager.cc:4808] TS 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783): delete failed for tablet 6a902c7087014cfe9747f7f1c612d110 because the tablet was not found. No further retry: Not found: Tablet not found: 6a902c7087014cfe9747f7f1c612d110
I20260809 08:47:58.055673 23538 raft_consensus.cc:482] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:47:58.055780 23538 raft_consensus.cc:504] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:58.055954 23538 leader_election.cc:288] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:58.056161 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428c6b3f791b462aade58d7af08053e4" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a" is_pre_election: true
I20260809 08:47:58.056160 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428c6b3f791b462aade58d7af08053e4" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734" is_pre_election: true
I20260809 08:47:58.056262 23110 raft_consensus.cc:2454] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:58.056263 23335 raft_consensus.cc:2454] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 0.
I20260809 08:47:58.056419 22782 leader_election.cc:302] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 9bdeb752d89541308553e5f5731f0734; no voters:
I20260809 08:47:58.056525 23538 raft_consensus.cc:2781] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:47:58.056576 23538 raft_consensus.cc:482] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:47:58.056591 23538 raft_consensus.cc:3037] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:58.057391 23538 raft_consensus.cc:504] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:58.057518 23538 leader_election.cc:288] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Requested vote from peers 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061), 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783)
I20260809 08:47:58.057683 23335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428c6b3f791b462aade58d7af08053e4" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ed79483d3344acfb4a3e0c6693d015a"
I20260809 08:47:58.057717 23110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "428c6b3f791b462aade58d7af08053e4" candidate_uuid: "059ce69a2989456c8bfcd4909ebffe9b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9bdeb752d89541308553e5f5731f0734"
I20260809 08:47:58.057765 23335 raft_consensus.cc:3037] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:58.057782 23110 raft_consensus.cc:3037] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:47:58.058336 23335 raft_consensus.cc:2454] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:58.058382 23110 raft_consensus.cc:2454] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 059ce69a2989456c8bfcd4909ebffe9b in term 1.
I20260809 08:47:58.058521 22784 leader_election.cc:302] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 059ce69a2989456c8bfcd4909ebffe9b, 1ed79483d3344acfb4a3e0c6693d015a; no voters:
I20260809 08:47:58.058648 23538 raft_consensus.cc:2781] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:47:58.058703 23538 raft_consensus.cc:686] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Becoming Leader. State: Replica: 059ce69a2989456c8bfcd4909ebffe9b, State: Running, Role: LEADER
I20260809 08:47:58.058755 23538 consensus_queue.cc:237] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } }
I20260809 08:47:58.059319 22636 catalog_manager.cc:5495] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b reported cstate change: term changed from 0 to 1, leader changed from <none> to 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65). New cstate: current_term: 1 leader_uuid: "059ce69a2989456c8bfcd4909ebffe9b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "059ce69a2989456c8bfcd4909ebffe9b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 37321 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 } health_report { overall_health: UNKNOWN } } }
I20260809 08:47:58.066599 23110 raft_consensus.cc:1261] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:58.066675 23335 raft_consensus.cc:1261] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Refusing update from remote peer 059ce69a2989456c8bfcd4909ebffe9b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:47:58.066780 23538 consensus_queue.cc:1035] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9bdeb752d89541308553e5f5731f0734" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 33061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:58.066864 23494 consensus_queue.cc:1035] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ed79483d3344acfb4a3e0c6693d015a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:47:58.075886 22636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:58.075968 22636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:47:58.076452 22636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d76f9fa0254d9fbb81c651dd42b0c1: Sending DeleteTablet for 3 replicas of tablet 428c6b3f791b462aade58d7af08053e4
I20260809 08:47:58.076670 23076 tablet_service.cc:1514] Processing DeleteTablet for tablet 428c6b3f791b462aade58d7af08053e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:58 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:60448
I20260809 08:47:58.076655 22852 tablet_service.cc:1514] Processing DeleteTablet for tablet 428c6b3f791b462aade58d7af08053e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:58 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:50886
I20260809 08:47:58.076751 23543 tablet_replica.cc:330] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: stopping tablet replica
I20260809 08:47:58.076751 23542 tablet_replica.cc:330] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: stopping tablet replica
I20260809 08:47:58.076793 23543 raft_consensus.cc:2229] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:47:58.076797 23542 raft_consensus.cc:2229] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:58.076795 23301 tablet_service.cc:1514] Processing DeleteTablet for tablet 428c6b3f791b462aade58d7af08053e4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:47:58 UTC) from {username='oryx', principal='oryx/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:36190
I20260809 08:47:58.076825 23542 raft_consensus.cc:2258] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:58.076843 23543 raft_consensus.cc:2258] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:58.076885 23484 tablet_replica.cc:330] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: stopping tablet replica
I20260809 08:47:58.076962 23484 raft_consensus.cc:2229] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:47:58.076992 23484 raft_consensus.cc:2258] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:47:58.077121 23543 ts_tablet_manager.cc:1905] T 428c6b3f791b462aade58d7af08053e4 P 059ce69a2989456c8bfcd4909ebffe9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:58.077168 23542 ts_tablet_manager.cc:1905] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:58.077258 23484 ts_tablet_manager.cc:1905] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:47:58.077260 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 22755
I20260809 08:47:58.078733 23542 ts_tablet_manager.cc:1918] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:58.078790 23542 log.cc:1196] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/428c6b3f791b462aade58d7af08053e4
I20260809 08:47:58.079020 23542 ts_tablet_manager.cc:1939] T 428c6b3f791b462aade58d7af08053e4 P 9bdeb752d89541308553e5f5731f0734: Deleting consensus metadata
I20260809 08:47:58.079358 22608 catalog_manager.cc:4841] TS 9bdeb752d89541308553e5f5731f0734 (127.10.184.66:33061): tablet 428c6b3f791b462aade58d7af08053e4 (table test-table [id=30462ab5765c44e1ba04ff93d2b6f593]) successfully deleted
I20260809 08:47:58.084125 23484 ts_tablet_manager.cc:1918] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 08:47:58.084188 23484 log.cc:1196] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/428c6b3f791b462aade58d7af08053e4
I20260809 08:47:58.084328 23484 ts_tablet_manager.cc:1939] T 428c6b3f791b462aade58d7af08053e4 P 1ed79483d3344acfb4a3e0c6693d015a: Deleting consensus metadata
I20260809 08:47:58.084583 22613 catalog_manager.cc:4841] TS 1ed79483d3344acfb4a3e0c6693d015a (127.10.184.67:34783): tablet 428c6b3f791b462aade58d7af08053e4 (table test-table [id=30462ab5765c44e1ba04ff93d2b6f593]) successfully deleted
W20260809 08:47:58.089694 22607 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:37321 (error 108)
I20260809 08:47:58.090011 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 22968
I20260809 08:47:58.093122 23581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:47:58.090097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:37321 (local address 127.0.0.1:50916)
0809 08:47:58.093034 (+ 2937us) negotiation.cc:107] Waiting for socket to connect
0809 08:47:58.093085 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:37321: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2916,"thread_start_us":76,"threads_started":1}
W20260809 08:47:58.093235 22607 catalog_manager.cc:4584] TS 059ce69a2989456c8bfcd4909ebffe9b (127.10.184.65:37321): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 428c6b3f791b462aade58d7af08053e4: Network error: Client connection negotiation failed: client connection to 127.10.184.65:37321: connect: Connection refused (error 111)
I20260809 08:47:58.107235 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 23195
I20260809 08:47:58.122983 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 22584
2026-08-09T08:47:58Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3062 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:58 dist-test-slave-ncp9 krb5kdc[23584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:58 dist-test-slave-ncp9 krb5kdc[23584](info): set up 2 sockets
Aug 09 08:47:58 dist-test-slave-ncp9 krb5kdc[23584](info): commencing operation
krb5kdc: starting...
W20260809 08:47:59.283236 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:47:59 dist-test-slave-ncp9 krb5kdc[23584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:47:59.391053 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36111
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:41477
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:36111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:47:59.472102 23603 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:47:59.472282 23603 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:47:59.472302 23603 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:47:59.473441 23603 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:47:59.473480 23603 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:47:59.473495 23603 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:47:59.473516 23603 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:47:59.473538 23603 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:47:59.474961 23603 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:41477
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:36111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:36111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23603
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:47:59.475178 23603 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:47:59.475368 23603 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:47:59.480077 23616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:47:59.480106 23614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:59.480120 23603 server_base.cc:1034] running on GCE node
W20260809 08:47:59.480091 23611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:47:59.480509 23603 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:47:59.480794 23603 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:47:59.481936 23603 hybrid_clock.cc:648] HybridClock initialized: now 1786265279481918 us; error 32 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T08:47:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1357 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:47:59 dist-test-slave-ncp9 krb5kdc[23625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:47:59 dist-test-slave-ncp9 krb5kdc[23625](info): set up 2 sockets
Aug 09 08:47:59 dist-test-slave-ncp9 krb5kdc[23625](info): commencing operation
krb5kdc: starting...
W20260809 08:48:00.602479 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:48:00 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:48:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:48:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:48:00.710719 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34389
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:34389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:00.789772 23646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:00.789963 23646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:00.789991 23646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:00.791082 23646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:48:00.791121 23646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:00.791141 23646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:00.791155 23646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:48:00.791170 23646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:48:00.792618 23646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:34389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:00.792860 23646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:00.793062 23646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:00.797650 23652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:00.797693 23654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:00.797819 23646 server_base.cc:1034] running on GCE node
W20260809 08:48:00.797701 23658 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:00.798106 23646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:00.798369 23646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:00.799507 23646 hybrid_clock.cc:648] HybridClock initialized: now 1786265280799492 us; error 29 us; skew 500 ppm
Aug 09 08:48:00 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265280, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:00.807240 23646 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:48:00.807617 23646 webserver.cc:460] Webserver started at http://127.10.184.126:39261/ using document root <none> and password file <none>
I20260809 08:48:00.807821 23646 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:00.807868 23646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:00.807989 23646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:00.809010 23646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "9358e67fddee41a4b0381c5140626499"
format_stamp: "Formatted at 2026-08-09 08:48:00 on dist-test-slave-ncp9"
server_key: "1f1f45e7446fbbb97c3936cd68923f92"
server_key_iv: "199585ef57feaf711b03a1379406b3a4"
server_key_version: "encryptionkey@0"
I20260809 08:48:00.809396 23646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "9358e67fddee41a4b0381c5140626499"
format_stamp: "Formatted at 2026-08-09 08:48:00 on dist-test-slave-ncp9"
server_key: "1f1f45e7446fbbb97c3936cd68923f92"
server_key_iv: "199585ef57feaf711b03a1379406b3a4"
server_key_version: "encryptionkey@0"
I20260809 08:48:00.810653 23646 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:48:00.811333 23670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:00.811478 23646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:00.811561 23646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9358e67fddee41a4b0381c5140626499"
format_stamp: "Formatted at 2026-08-09 08:48:00 on dist-test-slave-ncp9"
server_key: "1f1f45e7446fbbb97c3936cd68923f92"
server_key_iv: "199585ef57feaf711b03a1379406b3a4"
server_key_version: "encryptionkey@0"
I20260809 08:48:00.811618 23646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:00.848183 23646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:00.850204 23646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:00.850338 23646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:00.854000 23646 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:34389
I20260809 08:48:00.854048 23760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:34389 every 8 connection(s)
I20260809 08:48:00.854336 23646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:48:00.854851 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 23646
I20260809 08:48:00.854936 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:48:00.855118 10977 external_mini_cluster.cc:1428] Setting key 35356fcd6e45919356131ce742b815b8
I20260809 08:48:00.855225 23770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:00.857937 23770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Bootstrap starting.
I20260809 08:48:00.858490 23770 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:00.858790 23770 log.cc:824] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:00.859498 23770 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: No bootstrap required, opened a new log
Aug 09 08:48:00 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265280, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:00.861531 23770 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:00.861711 23770 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:00.861756 23770 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9358e67fddee41a4b0381c5140626499, State: Initialized, Role: FOLLOWER
I20260809 08:48:00.861886 23770 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:00.861968 23770 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:48:00.862000 23770 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:48:00.862046 23770 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:00.862679 23770 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:00.862785 23770 leader_election.cc:302] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9358e67fddee41a4b0381c5140626499; no voters:
I20260809 08:48:00.862949 23770 leader_election.cc:288] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:48:00.862999 23777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:48:00.863181 23770 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:48:00.863202 23777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 LEADER]: Becoming Leader. State: Replica: 9358e67fddee41a4b0381c5140626499, State: Running, Role: LEADER
I20260809 08:48:00.863365 23777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:00.863672 23777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9358e67fddee41a4b0381c5140626499. Latest consensus state: current_term: 1 leader_uuid: "9358e67fddee41a4b0381c5140626499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } } }
I20260809 08:48:00.863740 23777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:00.864043 23783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9358e67fddee41a4b0381c5140626499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } } }
I20260809 08:48:00.864122 23783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:00.864225 23788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:48:00.864678 23788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:48:00.865543 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:00.855587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55514 (local address 127.10.184.126:34389)
0809 08:48:00.855772 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:00.855780 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:00.855804 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:48:00.856243 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:00.856249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:00.856292 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:00.856406 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:00.857045 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:00.857895 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:00.858661 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:00.859196 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:00.859314 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:00.861585 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:00.861589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:00.861589 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:00.861600 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:00.863423 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:00.863815 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:00.863818 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:00.863818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:00.863918 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:00.864135 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:00.864137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:00.864137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:00.864456 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:00.865130 (+ 674us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:00.865214 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 08:48:00.865344 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260809 08:48:00.866808 23788 catalog_manager.cc:1347] Generated new cluster ID: 7652a5a58c114b278c9ecc3c656a3dae
I20260809 08:48:00.866863 23788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:48:00.880656 23788 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:48:00.881175 23788 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:48:00.891283 23788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Generated new TSK 0
I20260809 08:48:00.891453 23788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:48:00.925000 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:01.004240 23814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.004431 23814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.004462 23814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.005640 23814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.005679 23814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.005714 23814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:48:01.007136 23814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.007354 23814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.007552 23814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:01.012229 23826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.012332 23829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.012212 23823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.012405 23814 server_base.cc:1034] running on GCE node
I20260809 08:48:01.012596 23814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.012804 23814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.013969 23814 hybrid_clock.cc:648] HybridClock initialized: now 1786265281013963 us; error 41 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.016433 23814 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:48:01.016821 23814 webserver.cc:460] Webserver started at http://127.10.184.65:39767/ using document root <none> and password file <none>
I20260809 08:48:01.017031 23814 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.017079 23814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.017208 23814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:01.018177 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "87ed02cf93ba4a399c0a9ba9211218de"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a732fd4bc50e514260c50e1d9492e7d5"
server_key_iv: "d448bea6f8b8fe49d03ca40f01fad951"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.018554 23814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "87ed02cf93ba4a399c0a9ba9211218de"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a732fd4bc50e514260c50e1d9492e7d5"
server_key_iv: "d448bea6f8b8fe49d03ca40f01fad951"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.019841 23814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:48:01.020442 23843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.020602 23814 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.020684 23814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "87ed02cf93ba4a399c0a9ba9211218de"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a732fd4bc50e514260c50e1d9492e7d5"
server_key_iv: "d448bea6f8b8fe49d03ca40f01fad951"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.020742 23814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.031951 23814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.033555 23814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.033679 23814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.033867 23814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.034137 23814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.034179 23814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.034209 23814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.034231 23814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.039568 23814 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39819
I20260809 08:48:01.039633 24027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39819 every 8 connection(s)
I20260809 08:48:01.039901 23814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:01.047758 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.040534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:34683 (local address 127.10.184.126:34389)
0809 08:48:01.040592 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.040594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.040773 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:48:01.041264 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.041265 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.041274 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.041325 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.042167 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.042675 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.043551 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.043739 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.043781 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.045655 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:01.045657 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:01.045657 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:01.045664 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:01.046755 (+ 1091us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.047077 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.047078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.047078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.047120 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.047302 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.047302 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.047303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.047395 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:01.047439 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:01.047627 (+ 188us) server_negotiation.cc:299] Negotiation successful
0809 08:48:01.047693 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:48:01.047915 24031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.040491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:34683)
0809 08:48:01.040646 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.040694 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.041186 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.041377 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.041380 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.041911 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.042092 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.042096 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.042733 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.042734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.043491 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.043493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.043790 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.043790 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.043879 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.044256 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.044264 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.045567 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:01.046819 (+ 1252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.046825 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.046826 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.047024 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.047163 (+ 139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.047163 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.047163 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.047262 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.047435 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:01.047438 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:01.047538 (+ 100us) client_negotiation.cc:769] Sending connection context
0809 08:48:01.047610 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:48:01.047722 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260809 08:48:01.048354 24028 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34389
I20260809 08:48:01.048437 24028 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:01.048573 24028 heartbeater.cc:511] Master 127.10.184.126:34389 requested a full tablet report, sending...
I20260809 08:48:01.049055 23705 ts_manager.cc:194] Registered new tserver with Master: 87ed02cf93ba4a399c0a9ba9211218de (127.10.184.65:39819)
I20260809 08:48:01.049623 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:34683
I20260809 08:48:01.049811 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 23814
I20260809 08:48:01.049921 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:48:01.050096 10977 external_mini_cluster.cc:1428] Setting key 8d18d761ef247b684aef2437beb8cdff
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:48:01.072691 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:01.152343 24034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.152524 24034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.152545 24034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.153642 24034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.153684 24034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.153709 24034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:48:01.155043 24034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.155210 24034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.155373 24034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:48:01.160131 24034 server_base.cc:1034] running on GCE node
W20260809 08:48:01.160089 24043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.160079 24041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.160058 24045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.160495 24034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.160703 24034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.161835 24034 hybrid_clock.cc:648] HybridClock initialized: now 1786265281161818 us; error 28 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.164173 24034 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:48:01.164611 24034 webserver.cc:460] Webserver started at http://127.10.184.66:38093/ using document root <none> and password file <none>
I20260809 08:48:01.164845 24034 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.164913 24034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.165063 24034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:01.166015 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "4d429fb7c1054014a4996b93b44bf802"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a8a2749275b83a3cb624e5399279d7bd"
server_key_iv: "2b9db3ef1885ca6a7f4427585533b33d"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.166332 24034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "4d429fb7c1054014a4996b93b44bf802"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a8a2749275b83a3cb624e5399279d7bd"
server_key_iv: "2b9db3ef1885ca6a7f4427585533b33d"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.167485 24034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:48:01.168099 24055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.168241 24034 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:01.168322 24034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4d429fb7c1054014a4996b93b44bf802"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a8a2749275b83a3cb624e5399279d7bd"
server_key_iv: "2b9db3ef1885ca6a7f4427585533b33d"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.168385 24034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.177764 24034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.179330 24034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.179457 24034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.179641 24034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.179920 24034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.179961 24034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.179988 24034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.180012 24034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.185441 24034 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:33987
I20260809 08:48:01.185477 24241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:33987 every 8 connection(s)
I20260809 08:48:01.185756 24034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:48:01.187198 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 24034
I20260809 08:48:01.187317 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:48:01.187554 10977 external_mini_cluster.cc:1428] Setting key 82885eb85f9210169c0ecf13b853fd97
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:01.195122 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.186401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:45723 (local address 127.10.184.126:34389)
0809 08:48:01.186472 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.186474 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.186636 (+ 162us) server_negotiation.cc:407] Connection header received
0809 08:48:01.187123 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.187124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.187134 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.187184 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.188309 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.188832 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.190169 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.190455 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.190527 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.192418 (+ 1891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:01.192420 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:01.192420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:01.192428 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:01.193882 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.194235 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.194237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.194237 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.194300 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.194507 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.194508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.194508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.194623 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:01.194681 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:01.194946 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 08:48:01.195033 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:48:01.195261 24244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.186373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:45723)
0809 08:48:01.186526 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.186561 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.187022 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.187267 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.187271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.187981 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.188213 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.188220 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.188950 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.188951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.190072 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.190075 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.190529 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.190529 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.190621 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.191058 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.191067 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.192327 (+ 1260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:01.193956 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.193962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.193964 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.194165 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.194357 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.194358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.194358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.194442 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.194712 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:01.194718 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:01.194828 (+ 110us) client_negotiation.cc:769] Sending connection context
0809 08:48:01.194913 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:48:01.195059 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260809 08:48:01.195753 24242 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34389
I20260809 08:48:01.195834 24242 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:01.195996 24242 heartbeater.cc:511] Master 127.10.184.126:34389 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
I20260809 08:48:01.196367 23705 ts_manager.cc:194] Registered new tserver with Master: 4d429fb7c1054014a4996b93b44bf802 (127.10.184.66:33987)
I20260809 08:48:01.196779 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:45723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:48:01.211076 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:01.289247 24248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.289422 24248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.289443 24248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.290506 24248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.290537 24248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.290561 24248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:48:01.291904 24248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.292074 24248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.292248 24248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:01.296880 24267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.296850 24263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.297061 24248 server_base.cc:1034] running on GCE node
W20260809 08:48:01.296866 24265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.297304 24248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.297493 24248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.298620 24248 hybrid_clock.cc:648] HybridClock initialized: now 1786265281298602 us; error 35 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.300961 24248 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:48:01.301329 24248 webserver.cc:460] Webserver started at http://127.10.184.67:42477/ using document root <none> and password file <none>
I20260809 08:48:01.301508 24248 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.301548 24248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.301635 24248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:01.302544 24248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "7a2033b592034716b219e8b745e6035b"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "50b39e41df4b3476ffcf60eaa54bca67"
server_key_iv: "05dd2f78b9d98b31cd8f2c9ff13f8fcc"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.302882 24248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "7a2033b592034716b219e8b745e6035b"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "50b39e41df4b3476ffcf60eaa54bca67"
server_key_iv: "05dd2f78b9d98b31cd8f2c9ff13f8fcc"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.304127 24248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:48:01.304719 24277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.304868 24248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.304975 24248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "7a2033b592034716b219e8b745e6035b"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "50b39e41df4b3476ffcf60eaa54bca67"
server_key_iv: "05dd2f78b9d98b31cd8f2c9ff13f8fcc"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.305027 24248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.316551 24248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.318468 24248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.318596 24248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.318797 24248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.319106 24248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.319159 24248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.319209 24248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.319234 24248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.324559 24248 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:38223
I20260809 08:48:01.324595 24457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:38223 every 8 connection(s)
I20260809 08:48:01.324865 24248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:48:01.325214 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 24248
I20260809 08:48:01.325338 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:48:01.325531 10977 external_mini_cluster.cc:1428] Setting key 7a99b46bf5611e5cd5e54ac08f61e04d
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:01.333024 23773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.325921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:35749 (local address 127.10.184.126:34389)
0809 08:48:01.325966 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.325968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.326165 (+ 197us) server_negotiation.cc:407] Connection header received
0809 08:48:01.326677 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.326678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.326689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.326724 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.327547 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.328048 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.328972 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.329160 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.329207 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.330961 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:01.330963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:01.330963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:01.330970 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:01.332027 (+ 1057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.332347 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.332348 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.332348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.332391 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.332553 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.332553 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.332554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.332642 (+ 88us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:01.332691 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:01.332871 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 08:48:01.332962 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:48:01.333207 24461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.325894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:35749)
0809 08:48:01.326059 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.326093 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.326574 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.326783 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.326786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.327307 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.327486 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.327490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.328106 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.328106 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.328873 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.328875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.329216 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.329216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.329319 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.329682 (+ 363us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.329690 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.330886 (+ 1196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:01.332088 (+ 1202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.332094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.332095 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.332294 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.332432 (+ 138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:01.332432 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:01.332433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:01.332514 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:01.332685 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:01.332687 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:01.332784 (+ 97us) client_negotiation.cc:769] Sending connection context
0809 08:48:01.332854 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:48:01.333002 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":95,"threads_started":1}
I20260809 08:48:01.333593 24458 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34389
I20260809 08:48:01.333684 24458 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:01.333817 24458 heartbeater.cc:511] Master 127.10.184.126:34389 requested a full tablet report, sending...
I20260809 08:48:01.334129 23705 ts_manager.cc:194] Registered new tserver with Master: 7a2033b592034716b219e8b745e6035b (127.10.184.67:38223)
I20260809 08:48:01.334472 23705 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:35749
I20260809 08:48:01.337180 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
Entry for principal oryx/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab.
WARNING: no policy specified for oryx/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
Entry for principal oryx/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab.
WARNING: no policy specified for oryx/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
Entry for principal oryx/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab.
I20260809 08:48:01.397018 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 23814
I20260809 08:48:01.410100 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 24034
I20260809 08:48:01.417322 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 24248
I20260809 08:48:01.426038 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 23646
I20260809 08:48:01.434363 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34389
--webserver_interface=127.10.184.126
--webserver_port=39261
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:34389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:01.515110 24471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.515298 24471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.515318 24471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.516426 24471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:48:01.516466 24471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.516479 24471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.516502 24471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:48:01.516523 24471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:48:01.517982 24471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:34389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.518199 24471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.518395 24471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:01.523082 24483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.523073 24478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.523105 24471 server_base.cc:1034] running on GCE node
W20260809 08:48:01.523073 24480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.523497 24471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.523741 24471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.524871 24471 hybrid_clock.cc:648] HybridClock initialized: now 1786265281524860 us; error 26 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.527149 24471 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:48:01.527520 24471 webserver.cc:460] Webserver started at http://127.10.184.126:39261/ using document root <none> and password file <none>
I20260809 08:48:01.527694 24471 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.527743 24471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.528977 24471 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:48:01.529562 24498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.529731 24471 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:01.529812 24471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9358e67fddee41a4b0381c5140626499"
format_stamp: "Formatted at 2026-08-09 08:48:00 on dist-test-slave-ncp9"
server_key: "1f1f45e7446fbbb97c3936cd68923f92"
server_key_iv: "199585ef57feaf711b03a1379406b3a4"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.530095 24471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.553565 24471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.555217 24471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.555322 24471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.558972 24471 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:34389
I20260809 08:48:01.559026 24584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:34389 every 8 connection(s)
I20260809 08:48:01.559345 24471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:48:01.560758 24586 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:48:01.561474 24586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Bootstrap starting.
I20260809 08:48:01.568750 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 24471
I20260809 08:48:01.569113 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:39819
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:34389
--webserver_port=39767
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:48:01.569621 24586 log.cc:824] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:01.571566 24586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:48:01.571851 24586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Bootstrap complete.
I20260809 08:48:01.574849 24586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:01.575184 24586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9358e67fddee41a4b0381c5140626499, State: Initialized, Role: FOLLOWER
I20260809 08:48:01.575347 24586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:01.575420 24586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:48:01.575464 24586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:48:01.575501 24586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:48:01.576756 24586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:01.576865 24586 leader_election.cc:302] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9358e67fddee41a4b0381c5140626499; no voters:
I20260809 08:48:01.577039 24586 leader_election.cc:288] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:48:01.577186 24591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:48:01.577308 24586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:48:01.577509 24591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [term 2 LEADER]: Becoming Leader. State: Replica: 9358e67fddee41a4b0381c5140626499, State: Running, Role: LEADER
I20260809 08:48:01.577736 24591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } }
I20260809 08:48:01.578245 24593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9358e67fddee41a4b0381c5140626499. Latest consensus state: current_term: 2 leader_uuid: "9358e67fddee41a4b0381c5140626499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } } }
I20260809 08:48:01.578447 24593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:01.579111 24592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9358e67fddee41a4b0381c5140626499" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9358e67fddee41a4b0381c5140626499" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34389 } } }
I20260809 08:48:01.579186 24592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499 [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:01.579268 24613 catalog_manager.cc:1259] Loaded cluster ID: 7652a5a58c114b278c9ecc3c656a3dae
I20260809 08:48:01.579317 24613 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: loading cluster ID for follower catalog manager: success
I20260809 08:48:01.580523 24613 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: acquiring CA information for follower catalog manager: success
I20260809 08:48:01.580909 24613 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:48:01.589007 24618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:48:01.589236 24618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:48:01.589300 24618 catalog_manager.cc:1259] Loaded cluster ID: 7652a5a58c114b278c9ecc3c656a3dae
I20260809 08:48:01.589329 24618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:48:01.589499 24618 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:48:01.589565 24618 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9358e67fddee41a4b0381c5140626499: Loaded TSK: 0
I20260809 08:48:01.589704 24618 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:48:01.655640 24589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.655843 24589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.655874 24589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.657033 24589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.657073 24589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.657106 24589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:48:01.658496 24589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.65.keytab
--principal=oryx/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:39819
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=39767
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.658713 24589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.658903 24589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:01.663674 24626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.663822 24630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.663678 24628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.664305 24589 server_base.cc:1034] running on GCE node
I20260809 08:48:01.664505 24589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.664722 24589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.665854 24589 hybrid_clock.cc:648] HybridClock initialized: now 1786265281665841 us; error 34 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.668467 24589 init.cc:377] Logged in from keytab as oryx/127.10.184.65@KRBTEST.COM (short username oryx)
I20260809 08:48:01.669049 24589 webserver.cc:460] Webserver started at http://127.10.184.65:39767/ using document root <none> and password file <none>
I20260809 08:48:01.669296 24589 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.669377 24589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.671046 24589 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:48:01.671710 24642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.671872 24589 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:01.671984 24589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "87ed02cf93ba4a399c0a9ba9211218de"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a732fd4bc50e514260c50e1d9492e7d5"
server_key_iv: "d448bea6f8b8fe49d03ca40f01fad951"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.672389 24589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.694039 24589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.696214 24589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.696367 24589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.696605 24589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.696964 24589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.697011 24589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.697049 24589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.697077 24589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.704843 24589 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39819
I20260809 08:48:01.704878 24828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39819 every 8 connection(s)
I20260809 08:48:01.705328 24589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:48:01.706586 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 24589
I20260809 08:48:01.707055 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:33987
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:34389
--webserver_port=38093
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.715610 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.706192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:47665)
0809 08:48:01.706391 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.706439 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.707041 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.707630 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.707634 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.708262 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.708509 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.708516 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.710145 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.710147 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.711320 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.711323 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.712158 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.712160 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.712314 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.712948 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.712959 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.715388 (+ 2429us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":97,"threads_started":1}
W20260809 08:48:01.715662 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.706251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:47665 (local address 127.10.184.126:34389)
0809 08:48:01.707256 (+ 1005us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.707265 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.707300 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:48:01.707410 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.707414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.707455 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.707565 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.708654 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.710023 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.711433 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.712044 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.712594 (+ 550us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.715535 (+ 2941us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:47665: BlockingRecv error: recv got EOF from 127.10.184.65:47665 (error 108)
Metrics: {"server-negotiator.queue_time_us":948,"thread_start_us":96,"threads_started":1}
W20260809 08:48:01.715986 24829 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:34389 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.184.126:34389: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.720589 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.716207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:42003)
0809 08:48:01.716324 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.716335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.716388 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.716507 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.716508 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.716645 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.716696 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.716698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.717311 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.717312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.718087 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.718090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.718389 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.718390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.718465 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.718814 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.718819 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.720528 (+ 1709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99}
W20260809 08:48:01.720718 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.716274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:42003 (local address 127.10.184.126:34389)
0809 08:48:01.716327 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.716329 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.716373 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:48:01.716444 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.716444 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.716454 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.716500 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.716759 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.717242 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.718161 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.718332 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.718369 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.720615 (+ 2246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:42003: BlockingRecv error: recv got EOF from 127.10.184.65:42003 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.724545 24834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.720874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:54207)
0809 08:48:01.720933 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.720944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.720986 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.721091 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.721092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.721197 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.721225 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.721226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.721809 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.721810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.722293 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.722294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.722583 (+ 289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.722584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.722632 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.722974 (+ 342us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.722979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.724489 (+ 1510us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 08:48:01.724611 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.720958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:54207 (local address 127.10.184.126:34389)
0809 08:48:01.720996 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.720998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.721005 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:48:01.721027 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.721028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.721036 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.721079 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.721298 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.721746 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.722359 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.722529 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.722580 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.724569 (+ 1989us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:54207: BlockingRecv error: recv got EOF from 127.10.184.65:54207 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 08:48:01.724675 24829 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:48:01.790412 24838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.790607 24838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.790637 24838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.791735 24838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.791774 24838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.791807 24838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:48:01.793269 24838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.66.keytab
--principal=oryx/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:33987
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=38093
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.793483 24838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.793681 24838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:48:01.799021 24838 server_base.cc:1034] running on GCE node
W20260809 08:48:01.799082 24848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.798969 24847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.798985 24852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.799363 24838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.799552 24838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.800680 24838 hybrid_clock.cc:648] HybridClock initialized: now 1786265281800677 us; error 31 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.803071 24838 init.cc:377] Logged in from keytab as oryx/127.10.184.66@KRBTEST.COM (short username oryx)
I20260809 08:48:01.803517 24838 webserver.cc:460] Webserver started at http://127.10.184.66:38093/ using document root <none> and password file <none>
I20260809 08:48:01.803709 24838 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.803771 24838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.805038 24838 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:48:01.805557 24861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.805709 24838 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:01.805809 24838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4d429fb7c1054014a4996b93b44bf802"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "a8a2749275b83a3cb624e5399279d7bd"
server_key_iv: "2b9db3ef1885ca6a7f4427585533b33d"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.806100 24838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.815219 24838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.817203 24838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.817330 24838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.817529 24838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.817883 24838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.817921 24838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.817945 24838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.817960 24838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.823993 24838 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:33987
I20260809 08:48:01.824036 25073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:33987 every 8 connection(s)
I20260809 08:48:01.824330 24838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:48:01.831065 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.824957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:56045 (local address 127.10.184.126:34389)
0809 08:48:01.824996 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.824998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.825212 (+ 214us) server_negotiation.cc:407] Connection header received
0809 08:48:01.825684 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.825685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.825702 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.825752 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.826622 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.827115 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.828054 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.828259 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.828314 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.831006 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:56045: BlockingRecv error: recv got EOF from 127.10.184.66:56045 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:48:01.831079 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.824952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:56045)
0809 08:48:01.825093 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.825126 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.825584 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.825817 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.825821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.826344 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.826531 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.826536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.827199 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.827201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.827974 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.827976 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.828325 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.828326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.828442 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.828857 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.828865 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.830890 (+ 2025us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
W20260809 08:48:01.831370 25076 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:34389 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.184.126:34389: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
I20260809 08:48:01.833145 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 24838
I20260809 08:48:01.833477 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:38223
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:34389
--webserver_port=42477
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.836023 25081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.831561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:54613)
0809 08:48:01.831663 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.831675 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.831733 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.831859 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.831859 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.832035 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.832089 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.832091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.832721 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.832722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.833417 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.833420 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.833874 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.833876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.833948 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.834274 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.834279 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.835981 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":87}
W20260809 08:48:01.836133 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.831621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:54613 (local address 127.10.184.126:34389)
0809 08:48:01.831668 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.831670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.831714 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:48:01.831792 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.831793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.831803 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.831855 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.832165 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.832619 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.833519 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.833815 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.833869 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.836079 (+ 2210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:54613: BlockingRecv error: recv got EOF from 127.10.184.66:54613 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.840667 25083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.836232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:42583)
0809 08:48:01.836373 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.836384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.836430 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.836541 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.836542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.836699 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.836754 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.836756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.837380 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.837381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.838217 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.838220 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.838564 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.838565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.838626 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.838994 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.838999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.840612 (+ 1613us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
W20260809 08:48:01.840758 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.836400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:42583 (local address 127.10.184.126:34389)
0809 08:48:01.836441 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.836443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.836451 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:01.836473 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.836473 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.836481 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.836530 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.836820 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.837302 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.838293 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.838478 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.838534 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.840705 (+ 2171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:42583: BlockingRecv error: recv got EOF from 127.10.184.66:42583 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:48:01.840771 25076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:48:01.913795 25082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:01.913981 25082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:01.914001 25082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:01.915081 25082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:01.915113 25082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:01.915138 25082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:48:01.916468 25082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/krb5kdc/oryx_127.10.184.67.keytab
--principal=oryx/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:38223
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=42477
--tserver_master_addrs=127.10.184.126:34389
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:01.916636 25082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:01.916817 25082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:01.921600 25089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:01.921592 25090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.921782 25082 server_base.cc:1034] running on GCE node
W20260809 08:48:01.921873 25092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:01.922087 25082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:01.922331 25082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:01.923467 25082 hybrid_clock.cc:648] HybridClock initialized: now 1786265281923454 us; error 29 us; skew 500 ppm
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265281, etypes {rep=17 tkt=17 ses=17}, oryx/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:01.925858 25082 init.cc:377] Logged in from keytab as oryx/127.10.184.67@KRBTEST.COM (short username oryx)
I20260809 08:48:01.926326 25082 webserver.cc:460] Webserver started at http://127.10.184.67:42477/ using document root <none> and password file <none>
I20260809 08:48:01.926523 25082 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:01.926585 25082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:01.927878 25082 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:48:01.928447 25106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.928603 25082 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:01.928678 25082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "7a2033b592034716b219e8b745e6035b"
format_stamp: "Formatted at 2026-08-09 08:48:01 on dist-test-slave-ncp9"
server_key: "50b39e41df4b3476ffcf60eaa54bca67"
server_key_iv: "05dd2f78b9d98b31cd8f2c9ff13f8fcc"
server_key_version: "encryptionkey@0"
I20260809 08:48:01.929049 25082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:01.941296 25082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:01.942939 25082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:01.943060 25082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:01.943253 25082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:01.943526 25082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:01.943557 25082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.943593 25082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:01.943617 25082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:01.948832 25082 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:38223
I20260809 08:48:01.948916 25300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:38223 every 8 connection(s)
I20260809 08:48:01.949211 25082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 08:48:01.955853 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.949862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:39011 (local address 127.10.184.126:34389)
0809 08:48:01.949922 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.949924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.950110 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:48:01.950608 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.950609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.950623 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.950682 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.951577 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.952066 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.953051 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.953245 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.953299 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.955795 (+ 2496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:39011: BlockingRecv error: recv got EOF from 127.10.184.67:39011 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:48:01.955914 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.949860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:39011)
0809 08:48:01.949999 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.950033 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.950508 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.950750 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.950754 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.951301 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.951480 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.951485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.952144 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.952145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.952976 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.952978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.953311 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.953312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.953425 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.953855 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.953863 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.955704 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
W20260809 08:48:01.956101 25301 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:34389 (0 consecutive failures): Not authorized: Failed to ping master at 127.10.184.126:34389: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
I20260809 08:48:01.959009 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 25082
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.960172 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.956301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:45859)
0809 08:48:01.956339 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.956349 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.956380 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.956564 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.956565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.956682 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.956725 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.956727 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.957349 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.957350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.957833 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.957834 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.958144 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.958144 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.958213 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.958547 (+ 334us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.958551 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.960128 (+ 1577us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:48:01.960278 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.956432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:45859 (local address 127.10.184.126:34389)
0809 08:48:01.956482 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.956483 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.956489 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:48:01.956505 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.956506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.956513 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.956555 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.956788 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.957278 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.957896 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.958089 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.958140 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.960221 (+ 2081us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:45859: BlockingRecv error: recv got EOF from 127.10.184.67:45859 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:01 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:01.964350 25303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.960427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:57567)
0809 08:48:01.960539 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:01.960547 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:01.960588 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:01.960690 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:01.960691 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:01.960786 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:01.960809 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.960810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.961413 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.961414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.962074 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:01.962075 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.962385 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.962386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:01.962442 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.962720 (+ 278us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:01.962724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:01.964312 (+ 1588us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260809 08:48:01.964447 24836 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:01.960471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:57567 (local address 127.10.184.126:34389)
0809 08:48:01.960513 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.960514 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.960594 (+ 80us) server_negotiation.cc:407] Connection header received
0809 08:48:01.960634 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.960635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.960644 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.960682 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.960857 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.961349 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.962131 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.962322 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.962374 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.964398 (+ 2024us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:57567: BlockingRecv error: recv got EOF from 127.10.184.67:57567 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 08:48:01.964490 25301 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:48:01.965883 25307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:01.959367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55522 (local address 127.10.184.126:34389)
0809 08:48:01.959517 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:01.959519 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:01.959561 (+ 42us) server_negotiation.cc:407] Connection header received
0809 08:48:01.959629 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:01.959629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:01.959640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:01.959691 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:01.960308 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.960991 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.961911 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:01.962095 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:01.962151 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:01.963281 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:01.963285 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:01.963285 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:01.963294 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:01.964842 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.965144 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.965147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.965147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.965235 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:01.965363 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:01.965363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:01.965364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:01.965542 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:01.965637 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:01.965726 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 08:48:01.965802 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":72,"threads_started":1}
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:02.729360 25308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:02.725045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:33933)
0809 08:48:02.725168 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:02.725181 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:02.725244 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:02.725352 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:02.725352 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:02.725507 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:02.725557 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.725558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.726165 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.726166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.726730 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.726732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.727007 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.727008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.727042 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.727448 (+ 406us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:02.727452 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:02.729292 (+ 1840us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
W20260809 08:48:02.729444 25309 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:02.725098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:33933 (local address 127.10.184.126:34389)
0809 08:48:02.725242 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:02.725244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:02.725253 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:48:02.725279 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:02.725279 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:02.725289 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:02.725338 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:02.725604 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.726111 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.726781 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.726960 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.727009 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.729379 (+ 2370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:33933: BlockingRecv error: recv got EOF from 127.10.184.65:33933 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:02.845361 25310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:02.841107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:51117)
0809 08:48:02.841225 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:02.841236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:02.841293 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:02.841410 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:02.841410 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:02.841563 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:02.841611 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.841612 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.842233 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.842233 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.842766 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.842768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.843067 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.843067 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.843103 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.843540 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:02.843544 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:02.845309 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
W20260809 08:48:02.845459 25309 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:02.841152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:51117 (local address 127.10.184.126:34389)
0809 08:48:02.841219 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:02.841221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:02.841289 (+ 68us) server_negotiation.cc:407] Connection header received
0809 08:48:02.841344 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:02.841345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:02.841357 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:02.841400 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:02.841661 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.842162 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.842819 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.843013 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.843066 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.845400 (+ 2334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:51117: BlockingRecv error: recv got EOF from 127.10.184.66:51117 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:02 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:02.969012 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:02.964807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:48305)
0809 08:48:02.964948 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:02.964960 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:02.965022 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:02.965131 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:02.965132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:02.965290 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:02.965338 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.965340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.965928 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.965928 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.966472 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:02.966474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.966746 (+ 272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.966746 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:02.966783 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.967176 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:02.967181 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:02.968878 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":52,"threads_started":1}
W20260809 08:48:02.969055 25309 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:02.964915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48305 (local address 127.10.184.126:34389)
0809 08:48:02.964962 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:02.964964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:02.965006 (+ 42us) server_negotiation.cc:407] Connection header received
0809 08:48:02.965069 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:02.965070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:02.965081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:02.965115 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:02.965390 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.965877 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.966529 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:02.966700 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:02.966743 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:02.969001 (+ 2258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:48305: BlockingRecv error: recv got EOF from 127.10.184.67:48305 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:03.734242 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:03.729804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:40775)
0809 08:48:03.729921 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:03.729931 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:03.729981 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:03.730111 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:03.730112 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:03.730264 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:03.730313 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.730315 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.730935 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.730935 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.731450 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.731451 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.731769 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.731770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.731822 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.732261 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:03.732265 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:03.734166 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":66,"threads_started":1}
W20260809 08:48:03.734414 25316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:03.729855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40775 (local address 127.10.184.126:34389)
0809 08:48:03.729957 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:03.729959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:03.729968 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:48:03.730033 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:03.730034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:03.730047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:03.730102 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:03.730362 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.730870 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.731505 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.731701 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.731766 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.734266 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:40775: BlockingRecv error: recv got EOF from 127.10.184.65:40775 (error 108)
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":56,"threads_started":1}
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:03.849922 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:03.845812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:48975)
0809 08:48:03.845946 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:03.845957 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:03.846014 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:03.846130 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:03.846131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:03.846291 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:03.846342 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.846344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.846927 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.846928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.847449 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.847450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.847718 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.847719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.847755 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.848147 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:03.848151 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:03.849846 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
W20260809 08:48:03.850015 25316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:03.845869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:48975 (local address 127.10.184.126:34389)
0809 08:48:03.845920 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:03.845922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:03.846007 (+ 85us) server_negotiation.cc:407] Connection header received
0809 08:48:03.846055 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:03.846056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:03.846066 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:03.846115 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:03.846391 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.846875 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.847499 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.847674 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.847714 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.849956 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:48975: BlockingRecv error: recv got EOF from 127.10.184.66:48975 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:03 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:03.973517 25318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:03.969440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:38625)
0809 08:48:03.969581 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:03.969592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:03.969646 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:03.969773 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:03.969774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:03.969927 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:03.969976 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.969977 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.970567 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.970567 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.971094 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:03.971095 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.971363 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.971363 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:03.971399 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.971791 (+ 392us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:03.971795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:03.973459 (+ 1664us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
W20260809 08:48:03.973608 25316 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:03.969542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:38625 (local address 127.10.184.126:34389)
0809 08:48:03.969592 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:03.969594 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:03.969634 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:48:03.969704 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:03.969705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:03.969716 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:03.969760 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:03.970025 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.970512 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.971145 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:03.971318 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:03.971358 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:03.973544 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:38625: BlockingRecv error: recv got EOF from 127.10.184.67:38625 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:04.738720 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:04.734653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:53179)
0809 08:48:04.734773 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:04.734785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:04.734846 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:04.734969 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:04.734969 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:04.735120 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:04.735173 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.735175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.735775 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.735775 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.736297 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.736298 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.736573 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.736574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.736608 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.737043 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:04.737047 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:04.738628 (+ 1581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
W20260809 08:48:04.738795 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:04.734735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:53179 (local address 127.10.184.126:34389)
0809 08:48:04.734860 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:04.734862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:04.734881 (+ 19us) server_negotiation.cc:407] Connection header received
0809 08:48:04.734902 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:04.734902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:04.734912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:04.734957 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:04.735227 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.735724 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.736352 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.736531 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.736572 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.738734 (+ 2162us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:53179: BlockingRecv error: recv got EOF from 127.10.184.65:53179 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":66,"threads_started":1}
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:04.854493 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:04.850405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:43939)
0809 08:48:04.850528 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:04.850540 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:04.850599 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:04.850712 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:04.850713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:04.850869 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:04.850916 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.850917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.851516 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.851517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.852041 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.852042 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.852365 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.852365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.852402 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.852829 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:04.852834 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:04.854420 (+ 1586us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
W20260809 08:48:04.854648 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:04.850467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:43939 (local address 127.10.184.126:34389)
0809 08:48:04.850538 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:04.850540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:04.850594 (+ 54us) server_negotiation.cc:407] Connection header received
0809 08:48:04.850651 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:04.850652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:04.850663 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:04.850698 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:04.850964 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.851461 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.852107 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.852314 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.852364 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.854530 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:43939: BlockingRecv error: recv got EOF from 127.10.184.66:43939 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:04 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:04.978148 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:04.973902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:46887)
0809 08:48:04.974032 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:04.974041 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:04.974095 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:04.974205 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:04.974206 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:04.974360 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:04.974410 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.974412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.974999 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.975000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.975524 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:04.975525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.975856 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.975857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:04.975902 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.976301 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:04.976306 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:04.978072 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
W20260809 08:48:04.978221 25321 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:04.973958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:46887 (local address 127.10.184.126:34389)
0809 08:48:04.974006 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:04.974008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:04.974089 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:48:04.974135 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:04.974136 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:04.974149 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:04.974193 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:04.974455 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.974948 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.975573 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:04.975751 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:04.975828 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:04.978169 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:46887: BlockingRecv error: recv got EOF from 127.10.184.67:46887 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:05.743300 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:05.739157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:41323)
0809 08:48:05.739292 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:05.739303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:05.739353 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:05.739472 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:05.739473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:05.739631 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:05.739685 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.739687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.740280 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.740281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.740798 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.740799 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.741113 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.741113 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.741149 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.741552 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:05.741556 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:05.743223 (+ 1667us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
W20260809 08:48:05.743389 25325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:05.739222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:41323 (local address 127.10.184.126:34389)
0809 08:48:05.739335 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:05.739337 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:05.739345 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:05.739396 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:05.739396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:05.739407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:05.739460 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:05.739735 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.740230 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.740848 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.741067 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.741113 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.743327 (+ 2214us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:41323: BlockingRecv error: recv got EOF from 127.10.184.65:41323 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":59,"threads_started":1}
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:05.862243 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:05.854931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:35083)
0809 08:48:05.855058 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:05.855069 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:05.855125 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:05.855232 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:05.855232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:05.855384 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:05.855431 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.855433 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.856027 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.856027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.856543 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.856544 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.856821 (+ 277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.856822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.856858 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.860433 (+ 3575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:05.860438 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:05.862139 (+ 1701us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
W20260809 08:48:05.862316 25325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:05.854990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:35083 (local address 127.10.184.126:34389)
0809 08:48:05.855052 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:05.855054 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:05.855119 (+ 65us) server_negotiation.cc:407] Connection header received
0809 08:48:05.855179 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:05.855179 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:05.855190 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:05.855225 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:05.855477 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.855974 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.856596 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.856774 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.856815 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.862262 (+ 5447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:35083: BlockingRecv error: recv got EOF from 127.10.184.66:35083 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:05 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:05.982625 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:05.978563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:33619)
0809 08:48:05.978675 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:05.978685 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:05.978741 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:05.978875 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:05.978876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:05.979033 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:05.979081 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.979082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.979672 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.979672 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.980186 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:05.980187 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.980455 (+ 268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.980456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:05.980489 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.980885 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:05.980915 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:05.982565 (+ 1650us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":58,"threads_started":1}
W20260809 08:48:05.982823 25325 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:05.978602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:33619 (local address 127.10.184.126:34389)
0809 08:48:05.978684 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:05.978686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:05.978739 (+ 53us) server_negotiation.cc:407] Connection header received
0809 08:48:05.978791 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:05.978792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:05.978805 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:05.978846 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:05.979131 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.979615 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.980239 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:05.980410 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:05.980452 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:05.982761 (+ 2309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:33619: BlockingRecv error: recv got EOF from 127.10.184.67:33619 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:06.747924 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:06.743732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:50145)
0809 08:48:06.743878 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:06.743889 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:06.743941 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:06.744060 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:06.744061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:06.744229 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:06.744280 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.744282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.744886 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.744887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.745432 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.745433 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.745716 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.745716 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.745753 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.746197 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:06.746203 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:06.747847 (+ 1644us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
W20260809 08:48:06.748006 25329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:06.743817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:50145 (local address 127.10.184.126:34389)
0809 08:48:06.743930 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:06.743931 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:06.743940 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:48:06.743983 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:06.743983 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:06.743994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:06.744049 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:06.744334 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.744834 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.745484 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.745669 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.745711 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.747938 (+ 2227us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:50145: BlockingRecv error: recv got EOF from 127.10.184.65:50145 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:06.867034 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:06.862689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:47699)
0809 08:48:06.862832 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:06.862844 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:06.862891 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:06.863000 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:06.863001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:06.863163 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:06.863212 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.863213 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.863804 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.863804 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.864319 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.864320 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.864732 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.864733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.864784 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.865269 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:06.865275 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:06.866955 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":72,"threads_started":1}
W20260809 08:48:06.867106 25329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:06.862815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:47699 (local address 127.10.184.126:34389)
0809 08:48:06.862870 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:06.862871 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:06.862892 (+ 21us) server_negotiation.cc:407] Connection header received
0809 08:48:06.862939 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:06.862939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:06.862950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:06.862988 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:06.863257 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.863752 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.864372 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.864661 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.864722 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.867054 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:47699: BlockingRecv error: recv got EOF from 127.10.184.66:47699 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:06 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:06.987165 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:06.983015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:39097)
0809 08:48:06.983133 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:06.983142 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:06.983208 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:06.983319 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:06.983320 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:06.983477 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:06.983528 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.983530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.984112 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.984113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.984645 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:06.984647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.984946 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.984947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:06.984986 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.985440 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:06.985446 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:06.987097 (+ 1651us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
W20260809 08:48:06.987244 25329 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:06.983067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:39097 (local address 127.10.184.126:34389)
0809 08:48:06.983113 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:06.983114 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:06.983203 (+ 89us) server_negotiation.cc:407] Connection header received
0809 08:48:06.983248 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:06.983248 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:06.983261 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:06.983308 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:06.983576 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.984058 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.984697 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:06.984867 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:06.984940 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:06.987182 (+ 2242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:39097: BlockingRecv error: recv got EOF from 127.10.184.67:39097 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:07.752563 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:07.748322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:51327)
0809 08:48:07.748455 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:07.748465 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:07.748513 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:07.748632 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:07.748633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:07.748789 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:07.748838 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.748840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.749477 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.749477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.750023 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.750025 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.750323 (+ 298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.750323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.750360 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.750822 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:07.750826 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:07.752487 (+ 1661us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":61,"threads_started":1}
W20260809 08:48:07.752646 25335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:07.748388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:51327 (local address 127.10.184.126:34389)
0809 08:48:07.748502 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:07.748503 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:07.748513 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:07.748555 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:07.748556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:07.748565 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:07.748622 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:07.748906 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.749421 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.750079 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.750275 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.750326 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.752586 (+ 2260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:51327: BlockingRecv error: recv got EOF from 127.10.184.65:51327 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":58,"threads_started":1}
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:07.871784 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:07.867483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:52601)
0809 08:48:07.867602 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:07.867613 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:07.867678 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:07.867792 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:07.867793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:07.867944 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:07.867992 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.867993 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.868572 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.868573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.869176 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.869180 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.869616 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.869618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.869680 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.870119 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:07.870124 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:07.871728 (+ 1604us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
W20260809 08:48:07.871881 25335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:07.867533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:52601 (local address 127.10.184.126:34389)
0809 08:48:07.867599 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:07.867601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:07.867661 (+ 60us) server_negotiation.cc:407] Connection header received
0809 08:48:07.867724 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:07.867725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:07.867735 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:07.867780 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:07.868037 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.868521 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.869250 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.869543 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.869604 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.871815 (+ 2211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:52601: BlockingRecv error: recv got EOF from 127.10.184.66:52601 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:07 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:07.991814 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:07.987566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:36425)
0809 08:48:07.987699 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:07.987711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:07.987779 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:07.987901 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:07.987902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:07.988080 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:07.988128 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.988130 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.988776 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.988777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.989343 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:07.989345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.989664 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.989665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:07.989720 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.990121 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:07.990125 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:07.991737 (+ 1612us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":70,"threads_started":1}
W20260809 08:48:07.991894 25335 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:07.987645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:36425 (local address 127.10.184.126:34389)
0809 08:48:07.987702 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:07.987704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:07.987759 (+ 55us) server_negotiation.cc:407] Connection header received
0809 08:48:07.987828 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:07.987829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:07.987843 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:07.987890 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:07.988210 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.988706 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.989419 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:07.989606 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:07.989667 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:07.991839 (+ 2172us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:36425: BlockingRecv error: recv got EOF from 127.10.184.67:36425 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:08.757382 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:08.753161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:49839)
0809 08:48:08.753292 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:08.753303 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:08.753357 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:08.753483 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:08.753484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:08.753647 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:08.753695 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.753697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.754299 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.754299 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.754817 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.754818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.755087 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.755088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.755123 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.755571 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:08.755576 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:08.757304 (+ 1728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":67,"threads_started":1}
W20260809 08:48:08.757468 25341 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:08.753225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:49839 (local address 127.10.184.126:34389)
0809 08:48:08.753348 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:08.753350 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:08.753360 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:08.753399 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:08.753399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:08.753410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:08.753467 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:08.753747 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.754248 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.754868 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.755043 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.755081 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.757406 (+ 2325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:49839: BlockingRecv error: recv got EOF from 127.10.184.65:49839 (error 108)
Metrics: {"server-negotiator.queue_time_us":89,"thread_start_us":49,"threads_started":1}
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:08.876380 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:08.872194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:45695)
0809 08:48:08.872335 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:08.872345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:08.872404 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:08.872505 (+ 101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:08.872506 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:08.872652 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:08.872710 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.872712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.873326 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.873326 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.873850 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.873851 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.874124 (+ 273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.874126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.874172 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.874619 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:08.874624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:08.876306 (+ 1682us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
W20260809 08:48:08.876466 25341 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:08.872233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:45695 (local address 127.10.184.126:34389)
0809 08:48:08.872309 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:08.872311 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:08.872396 (+ 85us) server_negotiation.cc:407] Connection header received
0809 08:48:08.872444 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:08.872445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:08.872456 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:08.872500 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:08.872756 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.873273 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.873904 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.874075 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.874120 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.876407 (+ 2287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:45695: BlockingRecv error: recv got EOF from 127.10.184.66:45695 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:08 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:08.996503 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:08.992268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:57973)
0809 08:48:08.992392 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:08.992402 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:08.992450 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:08.992575 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:08.992575 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:08.992741 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:08.992797 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.992798 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.993488 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.993488 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.994019 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:08.994020 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.994367 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.994367 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:08.994405 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.994834 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:08.994838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:08.996444 (+ 1606us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":58,"threads_started":1}
W20260809 08:48:08.996590 25341 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:08.992367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:57973 (local address 127.10.184.126:34389)
0809 08:48:08.992421 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:08.992431 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:08.992442 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:48:08.992496 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:08.992497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:08.992510 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:08.992564 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:08.992864 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.993401 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.994104 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:08.994303 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:08.994366 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:08.996530 (+ 2164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:57973: BlockingRecv error: recv got EOF from 127.10.184.67:57973 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:09.762073 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:09.757846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:32881)
0809 08:48:09.757987 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:09.758001 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:09.758065 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:09.758219 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:09.758219 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:09.758382 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:09.758432 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.758434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.759020 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.759020 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.759539 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.759540 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.759816 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.759817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.759855 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:09.760296 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:09.760301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:09.762004 (+ 1703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":71,"threads_started":1}
W20260809 08:48:09.762171 25347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:09.757978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:32881 (local address 127.10.184.126:34389)
0809 08:48:09.758102 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:09.758104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:09.758110 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:48:09.758130 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:09.758131 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:09.758141 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:09.758206 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:09.758483 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.758970 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.759591 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.759769 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.759810 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:09.762096 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:32881: BlockingRecv error: recv got EOF from 127.10.184.65:32881 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:09.881445 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:09.876780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:34339)
0809 08:48:09.876937 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:09.876950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:09.877013 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:09.877238 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:09.877239 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:09.877454 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:09.877517 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.877519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.878286 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.878287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.878920 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.878922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.879232 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.879233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.879273 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:09.879659 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:09.879663 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:09.881367 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
W20260809 08:48:09.881515 25347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:09.876813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:34339 (local address 127.10.184.126:34389)
0809 08:48:09.876857 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:09.876859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:09.877002 (+ 143us) server_negotiation.cc:407] Connection header received
0809 08:48:09.877117 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:09.877118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:09.877133 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:09.877227 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:09.877576 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.878222 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.878978 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.879178 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.879225 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:09.881461 (+ 2236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:34339: BlockingRecv error: recv got EOF from 127.10.184.66:34339 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:09 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:10.001138 25350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:09.996972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:45241)
0809 08:48:09.997113 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:09.997124 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:09.997186 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:09.997289 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:09.997290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:09.997446 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:09.997495 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.997496 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.998077 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.998078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.998601 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:09.998602 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.998877 (+ 275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.998878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:09.998916 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:09.999324 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:09.999328 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:10.001057 (+ 1729us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":91,"threads_started":1}
W20260809 08:48:10.001210 25347 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:09.997001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:45241 (local address 127.10.184.126:34389)
0809 08:48:09.997047 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:09.997049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:09.997184 (+ 135us) server_negotiation.cc:407] Connection header received
0809 08:48:09.997227 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:09.997227 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:09.997241 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:09.997284 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:09.997541 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.998025 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.998657 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:09.998828 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:09.998868 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:10.001158 (+ 2290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:45241: BlockingRecv error: recv got EOF from 127.10.184.67:45241 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 09 08:48:10 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:10 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:10.766924 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:10.762521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:59333)
0809 08:48:10.762684 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:10.762696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:10.762761 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:10.762883 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:10.762883 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:10.763043 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:10.763095 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:10.763097 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.763695 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.763696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:10.764347 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:10.764351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.764720 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.764721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:10.764763 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:10.765171 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:10.765176 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:10.766866 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":92,"threads_started":1}
W20260809 08:48:10.767028 25355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:10.762582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:59333 (local address 127.10.184.126:34389)
0809 08:48:10.762707 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:10.762709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:10.762745 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:48:10.762813 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:10.762814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:10.762824 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:10.762869 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:10.763148 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.763642 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.764441 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.764662 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.764715 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:10.766952 (+ 2237us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:59333: BlockingRecv error: recv got EOF from 127.10.184.65:59333 (error 108)
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
Aug 09 08:48:10 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:10 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:10.892405 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:10.881830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:48697)
0809 08:48:10.881948 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:10.881959 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:10.882021 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:10.882132 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:10.882133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:10.882308 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:10.882359 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:10.882360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.882954 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.882955 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:10.883500 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:10.883502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.883793 (+ 291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.883794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:10.883832 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:10.890358 (+ 6526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:10.890364 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:10.892320 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":55,"threads_started":1}
W20260809 08:48:10.892469 25355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:10.881891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:48697 (local address 127.10.184.126:34389)
0809 08:48:10.881962 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:10.881964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:10.882003 (+ 39us) server_negotiation.cc:407] Connection header received
0809 08:48:10.882069 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:10.882070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:10.882081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:10.882125 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:10.882411 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.882897 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.883561 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:10.883742 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:10.883785 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:10.892421 (+ 8636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:48697: BlockingRecv error: recv got EOF from 127.10.184.66:48697 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 08:48:10 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:11 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:11.005927 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:11.001612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:58347)
0809 08:48:11.001759 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:11.001770 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:11.001829 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:11.001931 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:11.001932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:11.002102 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:11.002152 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.002154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.002738 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.002738 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.003253 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.003255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.003559 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.003561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.003607 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.004005 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:11.004009 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:11.005855 (+ 1846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
W20260809 08:48:11.005995 25355 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:11.001737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:58347 (local address 127.10.184.126:34389)
0809 08:48:11.001782 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:11.001783 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:11.001813 (+ 30us) server_negotiation.cc:407] Connection header received
0809 08:48:11.001868 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:11.001869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:11.001880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:11.001920 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:11.002203 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.002687 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.003308 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.003477 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.003519 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.005940 (+ 2421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:58347: BlockingRecv error: recv got EOF from 127.10.184.67:58347 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:48:11 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:11 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:11.771888 25361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:11.767399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:58105)
0809 08:48:11.767541 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:11.767553 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:11.767611 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:11.767746 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:11.767747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:11.767910 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:11.767960 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.767962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.768598 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.768599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.769254 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.769257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.769615 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.769616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.769673 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.770152 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:11.770157 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:11.771816 (+ 1659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":68,"threads_started":1}
W20260809 08:48:11.771974 25363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:11.767489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:58105 (local address 127.10.184.126:34389)
0809 08:48:11.767620 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:11.767622 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:11.767632 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:11.767655 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:11.767655 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:11.767673 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:11.767729 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:11.768015 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.768508 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.769342 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.769550 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.769605 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.771903 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:58105: BlockingRecv error: recv got EOF from 127.10.184.65:58105 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
Aug 09 08:48:11 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:11 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:11.897084 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:11.892886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:44845)
0809 08:48:11.893055 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:11.893066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:11.893132 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:11.893247 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:11.893248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:11.893405 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:11.893451 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.893453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.894046 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.894047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.894583 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:11.894585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.894863 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.894864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:11.894898 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.895328 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:11.895333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:11.897009 (+ 1676us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":88,"threads_started":1}
W20260809 08:48:11.897164 25363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:11.892973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:44845 (local address 127.10.184.126:34389)
0809 08:48:11.893025 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:11.893027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:11.893125 (+ 98us) server_negotiation.cc:407] Connection header received
0809 08:48:11.893186 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:11.893187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:11.893199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:11.893235 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:11.893500 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.893996 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.894636 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:11.894818 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:11.894859 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:11.897111 (+ 2252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:44845: BlockingRecv error: recv got EOF from 127.10.184.66:44845 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:12.010605 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:12.006355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:60013)
0809 08:48:12.006495 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:12.006508 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:12.006567 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:12.006700 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:12.006701 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:12.006855 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:12.006905 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.006907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.007503 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.007504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.008035 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.008037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.008334 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.008334 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.008375 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.008827 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:12.008831 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:12.010526 (+ 1695us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
W20260809 08:48:12.010685 25363 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:12.006484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:60013 (local address 127.10.184.126:34389)
0809 08:48:12.006548 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:12.006550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:12.006558 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:12.006618 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:12.006618 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:12.006632 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:12.006685 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:12.006952 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.007443 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.008097 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.008280 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.008327 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.010619 (+ 2292us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:60013: BlockingRecv error: recv got EOF from 127.10.184.67:60013 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:12.776646 25370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:12.772384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:49187)
0809 08:48:12.772504 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:12.772518 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:12.772571 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:12.772682 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:12.772683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:12.772847 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:12.772921 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.772923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.773557 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.773558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.774091 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.774092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.774425 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.774425 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.774460 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.774857 (+ 397us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:12.774861 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:12.776577 (+ 1716us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":56,"threads_started":1}
W20260809 08:48:12.776757 25371 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:12.772443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:49187 (local address 127.10.184.126:34389)
0809 08:48:12.772573 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:12.772575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:12.772600 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:48:12.772619 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:12.772620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:12.772630 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:12.772674 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:12.772989 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.773499 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.774164 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.774368 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.774420 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.776688 (+ 2268us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:49187: BlockingRecv error: recv got EOF from 127.10.184.65:49187 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:12 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:12.901888 25374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:12.897534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:38003)
0809 08:48:12.897659 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:12.897677 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:12.897738 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:12.897852 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:12.897853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:12.898033 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:12.898078 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.898079 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.898891 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.898892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.899446 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:12.899449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.899741 (+ 292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.899742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:12.899780 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.900173 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:12.900178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:12.901815 (+ 1637us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":58,"threads_started":1}
W20260809 08:48:12.902040 25371 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:12.897586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:38003 (local address 127.10.184.126:34389)
0809 08:48:12.897665 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:12.897667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:12.897730 (+ 63us) server_negotiation.cc:407] Connection header received
0809 08:48:12.897784 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:12.897785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:12.897796 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:12.897841 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:12.898144 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.898821 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.899511 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:12.899696 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:12.899738 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:12.901977 (+ 2239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:38003: BlockingRecv error: recv got EOF from 127.10.184.66:38003 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:13.015503 25375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:13.011030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:41189)
0809 08:48:13.011163 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:13.011187 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:13.011256 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:13.011389 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:13.011390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:13.011565 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:13.011614 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.011616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.012252 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.012254 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.012811 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.012813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.013173 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.013174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.013227 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.013614 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:13.013618 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:13.015434 (+ 1816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":67,"threads_started":1}
W20260809 08:48:13.015574 25371 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:13.011107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:41189 (local address 127.10.184.126:34389)
0809 08:48:13.011173 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:13.011175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:13.011247 (+ 72us) server_negotiation.cc:407] Connection header received
0809 08:48:13.011313 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:13.011313 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:13.011328 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:13.011374 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:13.011694 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.012181 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.012911 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.013103 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.013158 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.015514 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:41189: BlockingRecv error: recv got EOF from 127.10.184.67:41189 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:13.781541 25377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:13.777059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:48763)
0809 08:48:13.777180 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:13.777191 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:13.777240 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:13.777377 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:13.777378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:13.777564 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:13.777612 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.777614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.778301 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.778302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.778873 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.778879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.779241 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.779242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.779284 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.779707 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:13.779711 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:13.781461 (+ 1750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
W20260809 08:48:13.781657 25380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:13.777114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:48763 (local address 127.10.184.126:34389)
0809 08:48:13.777225 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:13.777227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:13.777237 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:13.777283 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:13.777301 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:13.777311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:13.777364 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:13.777667 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.778227 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.778973 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.779185 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.779235 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.781591 (+ 2356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:48763: BlockingRecv error: recv got EOF from 127.10.184.65:48763 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":56,"threads_started":1}
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:13 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:13.906531 25381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:13.902373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:32893)
0809 08:48:13.902556 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:13.902567 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:13.902626 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:13.902742 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:13.902742 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:13.902908 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:13.902952 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.902954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.903560 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.903561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.904079 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:13.904081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.904360 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.904360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:13.904398 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.904831 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:13.904835 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:13.906470 (+ 1635us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":87,"threads_started":1}
W20260809 08:48:13.906713 25380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:13.902480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:32893 (local address 127.10.184.126:34389)
0809 08:48:13.902532 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:13.902534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:13.902608 (+ 74us) server_negotiation.cc:407] Connection header received
0809 08:48:13.902668 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:13.902668 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:13.902679 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:13.902721 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:13.903002 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.903507 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.904133 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:13.904311 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:13.904355 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:13.906653 (+ 2298us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:32893: BlockingRecv error: recv got EOF from 127.10.184.66:32893 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:14.020244 25383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:14.015924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:34675)
0809 08:48:14.016050 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:14.016062 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:14.016126 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:14.016264 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:14.016264 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:14.016429 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:14.016484 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.016486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.017106 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.017107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.017637 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.017639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.017936 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.017937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.017974 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.018415 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:14.018420 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:14.020186 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
W20260809 08:48:14.020339 25380 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:14.015998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:34675 (local address 127.10.184.126:34389)
0809 08:48:14.016071 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:14.016073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:14.016117 (+ 44us) server_negotiation.cc:407] Connection header received
0809 08:48:14.016192 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:14.016193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:14.016206 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:14.016251 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:14.016534 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.017054 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.017698 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.017886 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.017931 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.020273 (+ 2342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:34675: BlockingRecv error: recv got EOF from 127.10.184.67:34675 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:14.786646 25389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:14.782021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:46119)
0809 08:48:14.782202 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:14.782214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:14.782274 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:14.782424 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:14.782425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:14.782608 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:14.782656 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.782658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.783313 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.783314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.783921 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.783923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.784276 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.784277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.784336 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.784762 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:14.784767 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:14.786571 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
W20260809 08:48:14.786742 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:14.782083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46119 (local address 127.10.184.126:34389)
0809 08:48:14.782227 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:14.782230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:14.782264 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:48:14.782339 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:14.782340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:14.782353 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:14.782409 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:14.782735 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.783239 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.784008 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.784211 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.784266 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.786660 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:46119: BlockingRecv error: recv got EOF from 127.10.184.65:46119 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:14 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:14.911358 25391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:14.906973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:50781)
0809 08:48:14.907095 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:14.907106 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:14.907185 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:14.907304 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:14.907305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:14.907466 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:14.907512 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.907513 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.908101 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.908102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.908687 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:14.908690 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.909042 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.909043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:14.909087 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.909574 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:14.909579 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:14.911299 (+ 1720us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":57,"threads_started":1}
W20260809 08:48:14.911456 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:14.907095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:50781 (local address 127.10.184.126:34389)
0809 08:48:14.907147 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:14.907149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:14.907159 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:14.907243 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:14.907243 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:14.907254 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:14.907297 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:14.907558 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.908048 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.908759 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:14.908989 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:14.909040 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:14.911397 (+ 2357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:50781: BlockingRecv error: recv got EOF from 127.10.184.66:50781 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:15.024742 25392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:15.020657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:51677)
0809 08:48:15.020778 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:15.020788 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:15.020846 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:15.020981 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:15.020981 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:15.021145 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:15.021197 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.021199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.021798 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.021799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.022332 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.022333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.022615 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.022616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.022662 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.023058 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:15.023062 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:15.024683 (+ 1621us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":68,"threads_started":1}
W20260809 08:48:15.024842 25390 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:15.020719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:51677 (local address 127.10.184.126:34389)
0809 08:48:15.020778 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:15.020779 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:15.020840 (+ 61us) server_negotiation.cc:407] Connection header received
0809 08:48:15.020884 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:15.020885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:15.020916 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:15.020968 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:15.021250 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.021745 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.022388 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.022565 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.022607 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.024780 (+ 2173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:51677: BlockingRecv error: recv got EOF from 127.10.184.67:51677 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:15.791313 25393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:15.787077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:42245)
0809 08:48:15.787212 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:15.787223 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:15.787273 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:15.787391 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:15.787392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:15.787556 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:15.787603 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.787605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.788222 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.788223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.788752 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.788753 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.789069 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.789070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.789115 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.789532 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:15.789536 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:15.791235 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
W20260809 08:48:15.791396 25395 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:15.787162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:42245 (local address 127.10.184.126:34389)
0809 08:48:15.787264 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:15.787265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:15.787273 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:15.787315 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:15.787316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:15.787326 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:15.787378 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:15.787658 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.788158 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.788805 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.789013 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.789058 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.791332 (+ 2274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:42245: BlockingRecv error: recv got EOF from 127.10.184.65:42245 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":52,"threads_started":1}
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:15 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:15.918610 25397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:15.911763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:44133)
0809 08:48:15.911895 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:15.911904 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:15.911960 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:15.912083 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:15.912083 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:15.912246 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:15.912292 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.912294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.913154 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.913155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.913684 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:15.913685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.913973 (+ 288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.913974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:15.914012 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.916840 (+ 2828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:15.916844 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:15.918551 (+ 1707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":70,"threads_started":1}
W20260809 08:48:15.918720 25395 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:15.911831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:44133 (local address 127.10.184.126:34389)
0809 08:48:15.911886 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:15.911888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:15.911954 (+ 66us) server_negotiation.cc:407] Connection header received
0809 08:48:15.912009 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:15.912009 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:15.912020 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:15.912071 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:15.912342 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.913085 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.913743 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:15.913926 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:15.913968 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:15.918638 (+ 4670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:44133: BlockingRecv error: recv got EOF from 127.10.184.66:44133 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.67@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:16.029697 25400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:16.025165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.67:36483)
0809 08:48:16.025291 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:16.025302 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:16.025370 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:16.025514 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:16.025515 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:16.025707 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:16.025758 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.025760 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.026399 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.026400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.026957 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.026959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.027290 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.027291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.027346 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.027759 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:16.027763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:16.029597 (+ 1834us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":63,"threads_started":1}
W20260809 08:48:16.029750 25395 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:16.025236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:36483 (local address 127.10.184.126:34389)
0809 08:48:16.025294 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:16.025297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:16.025350 (+ 53us) server_negotiation.cc:407] Connection header received
0809 08:48:16.025437 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:16.025438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:16.025453 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:16.025498 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:16.025831 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.026324 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.027036 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.027228 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.027283 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.029696 (+ 2413us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.67:36483: BlockingRecv error: recv got EOF from 127.10.184.67:36483 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.65@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:16.795851 25401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:16.791801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.65:37557)
0809 08:48:16.791941 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:16.791952 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:16.792004 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:16.792126 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:16.792127 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:16.792287 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:16.792338 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.792340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.792967 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.792968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.793505 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.793506 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.793787 (+ 281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.793787 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.793821 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.794208 (+ 387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:16.794212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:16.795783 (+ 1571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
W20260809 08:48:16.795941 25402 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:16.791870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:37557 (local address 127.10.184.126:34389)
0809 08:48:16.791981 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:16.791983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:16.791993 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:48:16.792058 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:16.792059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:16.792069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:16.792113 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:16.792388 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.792920 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.793558 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.793743 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.793783 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.795874 (+ 2091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.65:37557: BlockingRecv error: recv got EOF from 127.10.184.65:37557 (error 108)
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 08:48:16 dist-test-slave-ncp9 krb5kdc[23625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.10.184.66@KRBTEST.COM for oryx/127.10.184.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 08:48:16.923550 25405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:16.919105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34389 (local address 127.10.184.66:35159)
0809 08:48:16.919273 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:16.919284 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:16.919343 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:16.919459 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:16.919460 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:16.919622 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:16.919683 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.919685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.920286 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.920287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.920825 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:16.920827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.921137 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.921138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:16.921176 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.921583 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:16.921587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:16.923491 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.10.184.126:34389: Server oryx/127.10.184.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":100,"threads_started":1}
W20260809 08:48:16.923663 25402 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:16.919185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:35159 (local address 127.10.184.126:34389)
0809 08:48:16.919238 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:16.919240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:16.919336 (+ 96us) server_negotiation.cc:407] Connection header received
0809 08:48:16.919387 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:16.919388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:16.919399 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:16.919445 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:16.919731 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.920230 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.920886 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:16.921087 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:16.921134 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:16.923594 (+ 2460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.10.184.66:35159: BlockingRecv error: recv got EOF from 127.10.184.66:35159 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:48:16.966692 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 24589
I20260809 08:48:16.981057 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 24838
I20260809 08:48:16.989470 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 25082
I20260809 08:48:16.998968 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 24471
2026-08-09T08:48:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17517 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T08:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:48:17Z Disabled control of system clock
I20260809 08:48:17.032752 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37855
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:37855
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 08:48:17.111575 25412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.111759 25412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.111778 25412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.112870 25412 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:48:17.112921 25412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.112933 25412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.112944 25412 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:48:17.112955 25412 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:48:17.114228 25412 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:37855
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.114406 25412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.114622 25412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.119407 25420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.119505 25421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.119580 25423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.119432 25412 server_base.cc:1034] running on GCE node
I20260809 08:48:17.119840 25412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.120105 25412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.121238 25412 hybrid_clock.cc:648] HybridClock initialized: now 1786265297121226 us; error 27 us; skew 500 ppm
I20260809 08:48:17.122339 25412 webserver.cc:460] Webserver started at http://127.10.184.126:43829/ using document root <none> and password file <none>
I20260809 08:48:17.122531 25412 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.122576 25412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.122701 25412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.123677 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "caca2dca39e94f719c77f22c7770b80d"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e1f321373a480d38ab5e5e8c98d60854"
server_key_iv: "d22e84e295594e7caebff3a0409fefb3"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.124056 25412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "caca2dca39e94f719c77f22c7770b80d"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e1f321373a480d38ab5e5e8c98d60854"
server_key_iv: "d22e84e295594e7caebff3a0409fefb3"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.125259 25412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:48:17.125914 25432 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.126061 25412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:17.126143 25412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "caca2dca39e94f719c77f22c7770b80d"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e1f321373a480d38ab5e5e8c98d60854"
server_key_iv: "d22e84e295594e7caebff3a0409fefb3"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.126197 25412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.145444 25412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.145674 25412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.145769 25412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.149390 25412 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:37855
I20260809 08:48:17.149439 25533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:37855 every 8 connection(s)
I20260809 08:48:17.149724 25412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:48:17.150355 25534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:17.152810 25534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d: Bootstrap starting.
I20260809 08:48:17.153399 25534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:17.153673 25534 log.cc:824] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:17.154286 25534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d: No bootstrap required, opened a new log
I20260809 08:48:17.155809 25534 raft_consensus.cc:348] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } }
I20260809 08:48:17.155943 25534 raft_consensus.cc:374] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:17.155975 25534 raft_consensus.cc:729] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caca2dca39e94f719c77f22c7770b80d, State: Initialized, Role: FOLLOWER
I20260809 08:48:17.156098 25534 consensus_queue.cc:260] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } }
I20260809 08:48:17.156158 25534 raft_consensus.cc:388] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:48:17.156196 25534 raft_consensus.cc:482] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:48:17.156243 25534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:17.156790 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 25412
I20260809 08:48:17.156816 25534 raft_consensus.cc:504] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } }
I20260809 08:48:17.156876 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:48:17.156944 25534 leader_election.cc:302] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caca2dca39e94f719c77f22c7770b80d; no voters:
I20260809 08:48:17.157069 10977 external_mini_cluster.cc:1428] Setting key cbd90b1d10622712817474a6b2fc227e
I20260809 08:48:17.157092 25534 leader_election.cc:288] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:48:17.157135 25543 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:48:17.157300 25543 raft_consensus.cc:686] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [term 1 LEADER]: Becoming Leader. State: Replica: caca2dca39e94f719c77f22c7770b80d, State: Running, Role: LEADER
I20260809 08:48:17.157352 25534 sys_catalog.cc:564] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:48:17.157482 25543 consensus_queue.cc:237] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } }
I20260809 08:48:17.157932 25544 sys_catalog.cc:455] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caca2dca39e94f719c77f22c7770b80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } } }
I20260809 08:48:17.158017 25544 sys_catalog.cc:458] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:17.158002 25545 sys_catalog.cc:455] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [sys.catalog]: SysCatalogTable state changed. Reason: New leader caca2dca39e94f719c77f22c7770b80d. Latest consensus state: current_term: 1 leader_uuid: "caca2dca39e94f719c77f22c7770b80d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caca2dca39e94f719c77f22c7770b80d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37855 } } }
I20260809 08:48:17.158222 25545 sys_catalog.cc:458] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:17.158346 25558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:48:17.159426 25558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:48:17.160972 25558 catalog_manager.cc:1347] Generated new cluster ID: b2cfe4f06b81493bae0e8d36afee16a2
I20260809 08:48:17.161023 25558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:48:17.162684 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.157554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.10.184.126:37855)
0809 08:48:17.157747 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.157756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.157819 (+ 63us) server_negotiation.cc:407] Connection header received
0809 08:48:17.157968 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.157976 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.158044 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.158187 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.158460 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.159631 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.160553 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.161213 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.161338 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.161984 (+ 646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.161989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.161989 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.162212 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.162255 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.162271 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.162340 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.162396 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.162477 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260809 08:48:17.169250 25558 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:48:17.169905 25558 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:48:17.176966 25558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caca2dca39e94f719c77f22c7770b80d: Generated new TSK 0
I20260809 08:48:17.177125 25558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:48:17.180722 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--builtin_ntp_servers=127.10.184.84:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:48:17.260318 25579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.260507 25579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.260536 25579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.261668 25579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.261706 25579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.261739 25579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:48:17.263028 25579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25579
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.263254 25579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.263515 25579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.269193 25588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.269201 25591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.269201 25589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.269467 25579 server_base.cc:1034] running on GCE node
I20260809 08:48:17.269665 25579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.269871 25579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.271018 25579 hybrid_clock.cc:648] HybridClock initialized: now 1786265297271008 us; error 30 us; skew 500 ppm
I20260809 08:48:17.272182 25579 webserver.cc:460] Webserver started at http://127.10.184.65:33405/ using document root <none> and password file <none>
I20260809 08:48:17.272390 25579 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.272437 25579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.272555 25579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.273561 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "8f5d0689aa2b4391a759337ee901229c"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "d6f0cac9dee62d693f751e173e730ae8"
server_key_iv: "05a34963e85da732662cf2be857cac97"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.273962 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "8f5d0689aa2b4391a759337ee901229c"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "d6f0cac9dee62d693f751e173e730ae8"
server_key_iv: "05a34963e85da732662cf2be857cac97"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.275136 25579 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:48:17.275827 25612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.275993 25579 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:17.276075 25579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "8f5d0689aa2b4391a759337ee901229c"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "d6f0cac9dee62d693f751e173e730ae8"
server_key_iv: "05a34963e85da732662cf2be857cac97"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.276134 25579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.293916 25579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.294152 25579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.294267 25579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.294435 25579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:17.294713 25579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:17.294747 25579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.294772 25579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:17.294798 25579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.300125 25579 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:45739
I20260809 08:48:17.300185 25799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:45739 every 8 connection(s)
I20260809 08:48:17.300446 25579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:48:17.304783 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.301046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46597 (local address 127.10.184.126:37855)
0809 08:48:17.301088 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.301090 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.301320 (+ 230us) server_negotiation.cc:407] Connection header received
0809 08:48:17.301744 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.301745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.301756 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.301799 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.302384 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.302856 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.303743 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.303914 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.303954 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.304594 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.304596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.304596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.304599 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.304612 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.304616 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.304656 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.304705 (+ 49us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.304729 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:48:17.304989 25806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.301050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37855 (local address 127.10.184.65:46597)
0809 08:48:17.301216 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:17.301246 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:17.301672 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:17.301853 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:17.301856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:17.301864 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:17.302325 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.302329 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.302911 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.302911 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.303684 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.303686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.303962 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.303963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.304060 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.304481 (+ 421us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:17.304482 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:17.304508 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:17.304528 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:17.304655 (+ 127us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:17.304657 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:17.304659 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:48:17.304700 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:48:17.304771 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260809 08:48:17.305414 25800 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37855
I20260809 08:48:17.305497 25800 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:17.305665 25800 heartbeater.cc:511] Master 127.10.184.126:37855 requested a full tablet report, sending...
I20260809 08:48:17.306013 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 25579
I20260809 08:48:17.306113 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:48:17.306260 25460 ts_manager.cc:194] Registered new tserver with Master: 8f5d0689aa2b4391a759337ee901229c (127.10.184.65:45739)
I20260809 08:48:17.306301 10977 external_mini_cluster.cc:1428] Setting key fcdae0e3f4cc0743155f343d145920c2
I20260809 08:48:17.307042 25460 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.65:46597
I20260809 08:48:17.307448 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--builtin_ntp_servers=127.10.184.84:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 08:48:17.386686 25809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.386874 25809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.386894 25809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.387988 25809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.388026 25809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.388052 25809 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:48:17.389367 25809 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25809
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.389573 25809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.389856 25809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.394555 25820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.394599 25819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.394827 25809 server_base.cc:1034] running on GCE node
W20260809 08:48:17.394850 25824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.395066 25809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.395300 25809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.396441 25809 hybrid_clock.cc:648] HybridClock initialized: now 1786265297396422 us; error 32 us; skew 500 ppm
I20260809 08:48:17.397624 25809 webserver.cc:460] Webserver started at http://127.10.184.66:39931/ using document root <none> and password file <none>
I20260809 08:48:17.397827 25809 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.397876 25809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.398000 25809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.398975 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "86274e8e5aaa412899706958ed575d6f"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "7697d4100579331a09ce95bcc53d63b3"
server_key_iv: "2746d89c0ecd2f381adfd70878fd977f"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.399307 25809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "86274e8e5aaa412899706958ed575d6f"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "7697d4100579331a09ce95bcc53d63b3"
server_key_iv: "2746d89c0ecd2f381adfd70878fd977f"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.400478 25809 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:48:17.401126 25838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.401260 25809 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:17.401342 25809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "86274e8e5aaa412899706958ed575d6f"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "7697d4100579331a09ce95bcc53d63b3"
server_key_iv: "2746d89c0ecd2f381adfd70878fd977f"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.401400 25809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.413362 25809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.413583 25809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.413695 25809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.413872 25809 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:17.414139 25809 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:17.414178 25809 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.414204 25809 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:17.414229 25809 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.419342 25809 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43445
I20260809 08:48:17.419401 26015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43445 every 8 connection(s)
I20260809 08:48:17.419653 25809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:48:17.422027 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 25809
I20260809 08:48:17.422120 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:48:17.422308 10977 external_mini_cluster.cc:1428] Setting key 5cbdfe3a2f53193023e4bf96ef174999
I20260809 08:48:17.423141 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--builtin_ntp_servers=127.10.184.84:37051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 08:48:17.424479 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.420246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:34061 (local address 127.10.184.126:37855)
0809 08:48:17.420322 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.420324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.420490 (+ 166us) server_negotiation.cc:407] Connection header received
0809 08:48:17.420951 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.420952 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.420964 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.421009 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.421681 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.422197 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.423255 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.423455 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.423507 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.424226 (+ 719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.424227 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.424228 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.424232 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.424248 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.424252 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.424296 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.424372 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.424402 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:48:17.424665 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.420209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37855 (local address 127.10.184.66:34061)
0809 08:48:17.420384 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:17.420415 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:17.420841 (+ 426us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:17.421073 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:17.421076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:17.421084 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:17.421606 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.421611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.422276 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.422277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.423181 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.423185 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.423520 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.423521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.423643 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.424092 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:17.424093 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:17.424119 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:17.424140 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:17.424302 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:17.424307 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:17.424309 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:48:17.424369 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:48:17.424464 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260809 08:48:17.425134 26018 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37855
I20260809 08:48:17.425237 26018 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:17.425413 26018 heartbeater.cc:511] Master 127.10.184.126:37855 requested a full tablet report, sending...
I20260809 08:48:17.425743 25460 ts_manager.cc:194] Registered new tserver with Master: 86274e8e5aaa412899706958ed575d6f (127.10.184.66:43445)
I20260809 08:48:17.426105 25460 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.66:34061
W20260809 08:48:17.506039 26026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.506237 26026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.506256 26026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.507324 26026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.507357 26026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.507380 26026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:48:17.508670 26026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:37051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37855
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.508855 26026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.509121 26026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.513887 26039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.513913 26026 server_base.cc:1034] running on GCE node
W20260809 08:48:17.513885 26034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.513916 26032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.514245 26026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.514479 26026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.515609 26026 hybrid_clock.cc:648] HybridClock initialized: now 1786265297515600 us; error 26 us; skew 500 ppm
I20260809 08:48:17.516752 26026 webserver.cc:460] Webserver started at http://127.10.184.67:44239/ using document root <none> and password file <none>
I20260809 08:48:17.517002 26026 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.517053 26026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.517184 26026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.518168 26026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "855fdbaa11014f588c2e8a08e9fb42bf"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "1bb4fe62a7d7eefc014039032d641abf"
server_key_iv: "5b5869b640d6dcc9bdcbc624aa35c6ee"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.518500 26026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "855fdbaa11014f588c2e8a08e9fb42bf"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "1bb4fe62a7d7eefc014039032d641abf"
server_key_iv: "5b5869b640d6dcc9bdcbc624aa35c6ee"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.519675 26026 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:48:17.520300 26051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.520444 26026 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:17.520527 26026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "855fdbaa11014f588c2e8a08e9fb42bf"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "1bb4fe62a7d7eefc014039032d641abf"
server_key_iv: "5b5869b640d6dcc9bdcbc624aa35c6ee"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.520593 26026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.535240 26026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.535468 26026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.535580 26026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.535772 26026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:17.536054 26026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:17.536089 26026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.536116 26026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:17.536140 26026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.541399 26026 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:45805
I20260809 08:48:17.541457 26228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:45805 every 8 connection(s)
I20260809 08:48:17.541734 26026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:48:17.546219 25549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.542373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:39763 (local address 127.10.184.126:37855)
0809 08:48:17.542434 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.542436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.542614 (+ 178us) server_negotiation.cc:407] Connection header received
0809 08:48:17.543055 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.543056 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.543065 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.543115 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.543714 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.544201 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.545148 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.545324 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.545370 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.546048 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.546048 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.546049 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.546052 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.546061 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.546064 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.546089 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.546144 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.546162 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:48:17.546411 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.542329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37855 (local address 127.10.184.67:39763)
0809 08:48:17.542505 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:17.542542 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:17.542984 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:17.543167 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:17.543170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:17.543178 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:17.543644 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.543658 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.544261 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.544262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.545092 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:17.545095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:17.545380 (+ 285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:17.545381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:17.545477 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:17.545937 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:17.545938 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:17.545965 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:17.545986 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:17.546097 (+ 111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:17.546099 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:17.546101 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:48:17.546141 (+ 40us) client_negotiation.cc:240] Negotiation successful
0809 08:48:17.546213 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":82,"threads_started":1}
I20260809 08:48:17.546844 26229 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37855
I20260809 08:48:17.546919 26229 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:17.547073 26229 heartbeater.cc:511] Master 127.10.184.126:37855 requested a full tablet report, sending...
I20260809 08:48:17.547395 25460 ts_manager.cc:194] Registered new tserver with Master: 855fdbaa11014f588c2e8a08e9fb42bf (127.10.184.67:45805)
I20260809 08:48:17.547734 25460 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.67:39763
I20260809 08:48:17.548339 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 26026
I20260809 08:48:17.548416 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:48:17.548574 10977 external_mini_cluster.cc:1428] Setting key 319ed4488dfdc4d62b6a1329074e3095
I20260809 08:48:17.549330 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:48:17.551036 25549 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:17.550602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.10.184.126:37855)
0809 08:48:17.550671 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.550673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.550750 (+ 77us) server_negotiation.cc:407] Connection header received
0809 08:48:17.550807 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.550808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.550820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.550868 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.550983 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56492: BlockingRecv error: recv got EOF from 127.0.0.1:56492 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:48:17.551064 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 25579
I20260809 08:48:17.571085 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 25809
I20260809 08:48:17.577320 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 26026
I20260809 08:48:17.583508 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 25412
2026-08-09T08:48:17Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (580 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T08:48:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:48:17Z Disabled control of system clock
I20260809 08:48:17.612820 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:42593
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:42593
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:48:17.693333 26248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.693533 26248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.693553 26248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.694684 26248 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:48:17.694729 26248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.694741 26248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.694764 26248 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:48:17.694787 26248 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:48:17.696131 26248 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:42593
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:42593
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.696369 26248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.696655 26248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.701367 26257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.701434 26255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.701547 26248 server_base.cc:1034] running on GCE node
W20260809 08:48:17.701474 26260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.701853 26248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.702116 26248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.703251 26248 hybrid_clock.cc:648] HybridClock initialized: now 1786265297703237 us; error 29 us; skew 500 ppm
I20260809 08:48:17.704455 26248 webserver.cc:460] Webserver started at http://127.10.184.126:41131/ using document root <none> and password file <none>
I20260809 08:48:17.704679 26248 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.704725 26248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.704875 26248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.706017 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "0e699800c6244e75bfd50f134f6aba5a"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "4612ab2247aa5fe5ada2b26d3de11bb8"
server_key_iv: "3d58bb365984c4485a51375d46ea6363"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.706355 26248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "0e699800c6244e75bfd50f134f6aba5a"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "4612ab2247aa5fe5ada2b26d3de11bb8"
server_key_iv: "3d58bb365984c4485a51375d46ea6363"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.707567 26248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:48:17.708225 26270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.708377 26248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:17.708464 26248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "0e699800c6244e75bfd50f134f6aba5a"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "4612ab2247aa5fe5ada2b26d3de11bb8"
server_key_iv: "3d58bb365984c4485a51375d46ea6363"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.708527 26248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.711751 26248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.712001 26248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.712128 26248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.715830 26248 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:42593
I20260809 08:48:17.715862 26358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:42593 every 8 connection(s)
I20260809 08:48:17.716224 26248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:48:17.716821 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 26248
I20260809 08:48:17.716945 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:48:17.716992 26359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:17.717155 10977 external_mini_cluster.cc:1428] Setting key 6c3881086d8075cf8788984717cb3192
I20260809 08:48:17.719782 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.717741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43514 (local address 127.10.184.126:42593)
0809 08:48:17.717908 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.717914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.717937 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:48:17.718581 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.718587 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.718626 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.718746 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.719370 (+ 624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.719373 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.719374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.719377 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.719413 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.719423 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.719474 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.719549 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.719595 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
I20260809 08:48:17.720696 26359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a: Bootstrap starting.
I20260809 08:48:17.721277 26359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:17.721536 26359 log.cc:824] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:17.722162 26359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a: No bootstrap required, opened a new log
I20260809 08:48:17.723536 26359 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } }
I20260809 08:48:17.723676 26359 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:17.723714 26359 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e699800c6244e75bfd50f134f6aba5a, State: Initialized, Role: FOLLOWER
I20260809 08:48:17.723831 26359 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } }
I20260809 08:48:17.723893 26359 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:48:17.723924 26359 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:48:17.723968 26359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:17.724442 26359 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } }
I20260809 08:48:17.724536 26359 leader_election.cc:302] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e699800c6244e75bfd50f134f6aba5a; no voters:
I20260809 08:48:17.724697 26359 leader_election.cc:288] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:48:17.724997 26368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:48:17.725001 26359 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:48:17.725127 26368 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [term 1 LEADER]: Becoming Leader. State: Replica: 0e699800c6244e75bfd50f134f6aba5a, State: Running, Role: LEADER
I20260809 08:48:17.725250 26368 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } }
I20260809 08:48:17.725585 26368 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e699800c6244e75bfd50f134f6aba5a. Latest consensus state: current_term: 1 leader_uuid: "0e699800c6244e75bfd50f134f6aba5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } } }
I20260809 08:48:17.725589 26370 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e699800c6244e75bfd50f134f6aba5a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e699800c6244e75bfd50f134f6aba5a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 42593 } } }
I20260809 08:48:17.725764 26368 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:17.725811 26370 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:17.726208 26386 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:48:17.726698 26386 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:48:17.728083 26386 catalog_manager.cc:1347] Generated new cluster ID: 651be25e94f348348009f74518cf787f
I20260809 08:48:17.728129 26386 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:48:17.741065 26386 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:48:17.741212 26386 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:48:17.748744 26386 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e699800c6244e75bfd50f134f6aba5a: Generated new TSK 0
I20260809 08:48:17.748939 26386 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:48:17.750921 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--builtin_ntp_servers=127.10.184.84:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 08:48:17.838702 26404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.838896 26404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.838917 26404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.840034 26404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.840072 26404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.840096 26404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:48:17.841408 26404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.841637 26404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.841903 26404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.847621 26413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.847839 26404 server_base.cc:1034] running on GCE node
W20260809 08:48:17.847657 26410 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.847664 26409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.848186 26404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.848417 26404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.849553 26404 hybrid_clock.cc:648] HybridClock initialized: now 1786265297849540 us; error 31 us; skew 500 ppm
I20260809 08:48:17.850729 26404 webserver.cc:460] Webserver started at http://127.10.184.65:36699/ using document root <none> and password file <none>
I20260809 08:48:17.850941 26404 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.850998 26404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.851119 26404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.852101 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "a9d6e1441f454b71b6a8ef3538597663"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e8ecc15281239be3677a0820a23295fd"
server_key_iv: "beab34b585042d1543571674145a854d"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.852418 26404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "a9d6e1441f454b71b6a8ef3538597663"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e8ecc15281239be3677a0820a23295fd"
server_key_iv: "beab34b585042d1543571674145a854d"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.853796 26404 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:48:17.854511 26425 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.854656 26404 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:17.854741 26404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "a9d6e1441f454b71b6a8ef3538597663"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "e8ecc15281239be3677a0820a23295fd"
server_key_iv: "beab34b585042d1543571674145a854d"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.854800 26404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.857965 26404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.858201 26404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.858314 26404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.858497 26404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:17.858762 26404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:17.858794 26404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.858819 26404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:17.858844 26404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.864022 26404 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:46153
I20260809 08:48:17.864084 26602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:46153 every 8 connection(s)
I20260809 08:48:17.864323 26404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:48:17.866510 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 26404
I20260809 08:48:17.866611 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:48:17.866823 10977 external_mini_cluster.cc:1428] Setting key c2c6eb78ab09b1c94d50220a8818bfd7
I20260809 08:48:17.867347 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.864940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:50175 (local address 127.10.184.126:42593)
0809 08:48:17.864983 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.864985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.865204 (+ 219us) server_negotiation.cc:407] Connection header received
0809 08:48:17.865743 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.865745 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.865752 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.865802 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.867028 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.867030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.867031 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.867036 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.867055 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.867059 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.867119 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.867245 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.867260 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:48:17.867493 26605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.864928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42593 (local address 127.10.184.65:50175)
0809 08:48:17.865098 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:17.865129 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:17.865631 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:17.865866 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:17.865870 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:17.865879 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:17.866867 (+ 988us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:17.866868 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:17.866904 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:17.866938 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:17.867150 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:17.867159 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:17.867162 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:48:17.867232 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:48:17.867295 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260809 08:48:17.868014 26603 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42593
I20260809 08:48:17.868227 26603 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:17.868388 26603 heartbeater.cc:511] Master 127.10.184.126:42593 requested a full tablet report, sending...
I20260809 08:48:17.868549 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--builtin_ntp_servers=127.10.184.84:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 08:48:17.868886 26296 ts_manager.cc:194] Registered new tserver with Master: a9d6e1441f454b71b6a8ef3538597663 (127.10.184.65:46153)
W20260809 08:48:17.948390 26607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:17.948577 26607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:17.948597 26607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:17.949685 26607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:17.949718 26607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:17.949741 26607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:48:17.951004 26607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:17.951191 26607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:17.951421 26607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:17.956029 26621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.956053 26616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:17.956271 26623 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:17.956243 26607 server_base.cc:1034] running on GCE node
I20260809 08:48:17.956478 26607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:17.956689 26607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:17.957836 26607 hybrid_clock.cc:648] HybridClock initialized: now 1786265297957816 us; error 33 us; skew 500 ppm
I20260809 08:48:17.958987 26607 webserver.cc:460] Webserver started at http://127.10.184.66:34901/ using document root <none> and password file <none>
I20260809 08:48:17.959209 26607 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:17.959252 26607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:17.959376 26607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:17.960357 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "12afec4b0ea84b5da2a6862042eda318"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "0d04010b010269e5fb3cf73ede4d885d"
server_key_iv: "58b75df02026134c1cb0892474c57d31"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.960681 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "12afec4b0ea84b5da2a6862042eda318"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "0d04010b010269e5fb3cf73ede4d885d"
server_key_iv: "58b75df02026134c1cb0892474c57d31"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.961952 26607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:48:17.962580 26635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.962726 26607 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:17.962802 26607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "12afec4b0ea84b5da2a6862042eda318"
format_stamp: "Formatted at 2026-08-09 08:48:17 on dist-test-slave-ncp9"
server_key: "0d04010b010269e5fb3cf73ede4d885d"
server_key_iv: "58b75df02026134c1cb0892474c57d31"
server_key_version: "encryptionkey@0"
I20260809 08:48:17.962869 26607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:17.965956 26607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:17.966177 26607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:17.966288 26607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:17.966460 26607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:17.966733 26607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:17.966771 26607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.966795 26607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:17.966812 26607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:17.972028 26607 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:46169
I20260809 08:48:17.972083 26804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:46169 every 8 connection(s)
I20260809 08:48:17.972365 26607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:48:17.973470 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 26607
I20260809 08:48:17.973569 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:48:17.973753 10977 external_mini_cluster.cc:1428] Setting key 272e2b212b2843cfd116dd14f467a277
I20260809 08:48:17.974735 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--builtin_ntp_servers=127.10.184.84:39807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 08:48:17.975399 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.973016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:57411 (local address 127.10.184.126:42593)
0809 08:48:17.973057 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:17.973059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:17.973354 (+ 295us) server_negotiation.cc:407] Connection header received
0809 08:48:17.973890 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:17.973891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:17.973897 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:17.973960 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:17.975202 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:17.975203 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:17.975203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:17.975207 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:17.975220 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:17.975224 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:17.975269 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:17.975336 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:48:17.975346 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:48:17.975589 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:17.973030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42593 (local address 127.10.184.66:57411)
0809 08:48:17.973235 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:17.973265 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:17.973790 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:17.974012 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:17.974016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:17.974025 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:17.975049 (+ 1024us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:17.975050 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:17.975084 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:17.975115 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:17.975271 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:17.975274 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:17.975277 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:48:17.975344 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:48:17.975402 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
I20260809 08:48:17.975998 26805 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42593
I20260809 08:48:17.976080 26805 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:17.976223 26805 heartbeater.cc:511] Master 127.10.184.126:42593 requested a full tablet report, sending...
I20260809 08:48:17.976568 26296 ts_manager.cc:194] Registered new tserver with Master: 12afec4b0ea84b5da2a6862042eda318 (127.10.184.66:46169)
W20260809 08:48:18.054147 26810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:18.054340 26810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:18.054360 26810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:18.055428 26810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:18.055459 26810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:18.055483 26810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:48:18.056739 26810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:42593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:18.056964 26810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:18.057215 26810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:18.062016 26817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:18.062074 26810 server_base.cc:1034] running on GCE node
W20260809 08:48:18.062044 26819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:18.062044 26815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:18.062379 26810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:18.062593 26810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:18.063731 26810 hybrid_clock.cc:648] HybridClock initialized: now 1786265298063721 us; error 32 us; skew 500 ppm
I20260809 08:48:18.064837 26810 webserver.cc:460] Webserver started at http://127.10.184.67:41993/ using document root <none> and password file <none>
I20260809 08:48:18.065075 26810 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:18.065125 26810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:18.065260 26810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:18.066220 26810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "a793a2435ce242c8b1df0b3d4c604606"
format_stamp: "Formatted at 2026-08-09 08:48:18 on dist-test-slave-ncp9"
server_key: "ad5734e11a20ee7b2aa05e7cc6d77363"
server_key_iv: "b73ad46070d93cf3c4d37f3d75f5539d"
server_key_version: "encryptionkey@0"
I20260809 08:48:18.066542 26810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "a793a2435ce242c8b1df0b3d4c604606"
format_stamp: "Formatted at 2026-08-09 08:48:18 on dist-test-slave-ncp9"
server_key: "ad5734e11a20ee7b2aa05e7cc6d77363"
server_key_iv: "b73ad46070d93cf3c4d37f3d75f5539d"
server_key_version: "encryptionkey@0"
I20260809 08:48:18.067766 26810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:48:18.068382 26828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:18.068526 26810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:18.068620 26810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "a793a2435ce242c8b1df0b3d4c604606"
format_stamp: "Formatted at 2026-08-09 08:48:18 on dist-test-slave-ncp9"
server_key: "ad5734e11a20ee7b2aa05e7cc6d77363"
server_key_iv: "b73ad46070d93cf3c4d37f3d75f5539d"
server_key_version: "encryptionkey@0"
I20260809 08:48:18.068681 26810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:18.071862 26810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:18.072083 26810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:18.072225 26810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:18.072405 26810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:18.072669 26810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:18.072700 26810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:18.072724 26810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:18.072749 26810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:18.078011 26810 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:41341
I20260809 08:48:18.078068 27009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:41341 every 8 connection(s)
I20260809 08:48:18.078325 26810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:48:18.079788 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 26810
I20260809 08:48:18.079881 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:48:18.080045 10977 external_mini_cluster.cc:1428] Setting key 877d1ecb300ac451008a7456ecfd5949
I20260809 08:48:18.080940 26364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:18.078912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:40875 (local address 127.10.184.126:42593)
0809 08:48:18.078985 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:18.078987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:18.079137 (+ 150us) server_negotiation.cc:407] Connection header received
0809 08:48:18.079532 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:18.079533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:18.079538 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:18.079585 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:18.080707 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:18.080708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:18.080709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:48:18.080712 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:18.080727 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:48:18.080730 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:18.080772 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:18.080839 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:48:18.080851 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:48:18.081107 27016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:18.078883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:42593 (local address 127.10.184.67:40875)
0809 08:48:18.079035 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:18.079065 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:18.079460 (+ 395us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:18.079633 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:18.079636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:18.079644 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:18.080559 (+ 915us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:48:18.080559 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:18.080587 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:48:18.080621 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:18.080785 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:18.080789 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:18.080791 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:48:18.080843 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:48:18.080916 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":76,"threads_started":1}
I20260809 08:48:18.081497 27013 heartbeater.cc:347] Connected to a master server at 127.10.184.126:42593
I20260809 08:48:18.081580 27013 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:18.081692 27013 heartbeater.cc:511] Master 127.10.184.126:42593 requested a full tablet report, sending...
I20260809 08:48:18.081995 26296 ts_manager.cc:194] Registered new tserver with Master: a793a2435ce242c8b1df0b3d4c604606 (127.10.184.67:41341)
I20260809 08:48:18.091583 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 08:48:18.093261 26364 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:48:18.092815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43522 (local address 127.10.184.126:42593)
0809 08:48:18.092855 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:18.092857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:18.092989 (+ 132us) server_negotiation.cc:407] Connection header received
0809 08:48:18.093040 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:18.093041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:18.093045 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:18.093088 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:18.093200 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43522: BlockingRecv error: recv got EOF from 127.0.0.1:43522 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:48:18.093317 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 26404
I20260809 08:48:18.101729 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 26607
I20260809 08:48:18.113714 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 26810
I20260809 08:48:18.122567 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 26248
2026-08-09T08:48:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (540 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:48:18 dist-test-slave-ncp9 krb5kdc[27031](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:48:18 dist-test-slave-ncp9 krb5kdc[27031](info): set up 2 sockets
Aug 09 08:48:18 dist-test-slave-ncp9 krb5kdc[27031](info): commencing operation
krb5kdc: starting...
W20260809 08:48:19.249397 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:48:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:48:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:48:19.358768 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44979
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:19.437227 27052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:19.437412 27052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:19.437440 27052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:19.438522 27052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:48:19.438563 27052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:19.438581 27052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:19.438598 27052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:48:19.438613 27052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:48:19.440044 27052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:19.440270 27052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:19.440470 27052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:19.445907 27060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.445984 27064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.445946 27061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:19.446249 27052 server_base.cc:1034] running on GCE node
I20260809 08:48:19.446508 27052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:19.446827 27052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:19.447988 27052 hybrid_clock.cc:648] HybridClock initialized: now 1786265299447971 us; error 43 us; skew 500 ppm
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:19.451014 27052 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:48:19.451501 27052 webserver.cc:460] Webserver started at http://127.10.184.126:46019/ using document root <none> and password file <none>
I20260809 08:48:19.451761 27052 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:19.451815 27052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:19.451956 27052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:19.453027 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "c672146fdcba4b2c980791297dbec26d"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "eef0fbf178c80616624e0b0a71ab9690"
server_key_iv: "3ace44ba1d432903196db99d689d7af1"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.453359 27052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "c672146fdcba4b2c980791297dbec26d"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "eef0fbf178c80616624e0b0a71ab9690"
server_key_iv: "3ace44ba1d432903196db99d689d7af1"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.454548 27052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:48:19.455207 27072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.455359 27052 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:48:19.455441 27052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "c672146fdcba4b2c980791297dbec26d"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "eef0fbf178c80616624e0b0a71ab9690"
server_key_iv: "3ace44ba1d432903196db99d689d7af1"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.455506 27052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:19.473233 27052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:19.475226 27052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:19.475356 27052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:19.479014 27052 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44979
I20260809 08:48:19.479046 27155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44979 every 8 connection(s)
I20260809 08:48:19.479398 27052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:48:19.480329 27156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:19.482486 27156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: Bootstrap starting.
I20260809 08:48:19.482764 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 27052
I20260809 08:48:19.482858 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:48:19.483019 10977 external_mini_cluster.cc:1428] Setting key c4dad1db52e22c3c486421205b81bcba
I20260809 08:48:19.483047 27156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:19.483320 27156 log.cc:824] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:19.484053 27156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: No bootstrap required, opened a new log
I20260809 08:48:19.486236 27156 raft_consensus.cc:348] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } }
I20260809 08:48:19.486399 27156 raft_consensus.cc:374] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:19.486446 27156 raft_consensus.cc:729] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c672146fdcba4b2c980791297dbec26d, State: Initialized, Role: FOLLOWER
I20260809 08:48:19.486568 27156 consensus_queue.cc:260] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } }
I20260809 08:48:19.486665 27156 raft_consensus.cc:388] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:48:19.486703 27156 raft_consensus.cc:482] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:48:19.486750 27156 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:19.487294 27156 raft_consensus.cc:504] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } }
I20260809 08:48:19.487398 27156 leader_election.cc:302] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c672146fdcba4b2c980791297dbec26d; no voters:
I20260809 08:48:19.487541 27156 leader_election.cc:288] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:48:19.487581 27162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:48:19.487744 27162 raft_consensus.cc:686] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [term 1 LEADER]: Becoming Leader. State: Replica: c672146fdcba4b2c980791297dbec26d, State: Running, Role: LEADER
I20260809 08:48:19.487773 27156 sys_catalog.cc:564] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:48:19.487841 27162 consensus_queue.cc:237] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } }
I20260809 08:48:19.488143 27162 sys_catalog.cc:455] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c672146fdcba4b2c980791297dbec26d. Latest consensus state: current_term: 1 leader_uuid: "c672146fdcba4b2c980791297dbec26d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } } }
I20260809 08:48:19.488219 27162 sys_catalog.cc:458] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:19.488386 27165 sys_catalog.cc:455] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c672146fdcba4b2c980791297dbec26d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c672146fdcba4b2c980791297dbec26d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44979 } } }
I20260809 08:48:19.488451 27165 sys_catalog.cc:458] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d [sys.catalog]: This master's current role is: LEADER
I20260809 08:48:19.488554 27172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:48:19.489028 27172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:19.491070 27172 catalog_manager.cc:1347] Generated new cluster ID: c658b53be2d44e018fcb73c2c25b3b27
I20260809 08:48:19.491114 27172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:48:19.494760 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:19.483481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.10.184.126:44979)
0809 08:48:19.483639 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:19.483648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:19.483673 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:48:19.484211 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:19.484219 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:19.484277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:19.484411 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:19.484985 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.486323 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.487131 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.487730 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.487834 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:19.491275 (+ 3441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:19.491279 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:19.491279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:19.491290 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:19.493189 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.493617 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.493620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.493621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.493732 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.493934 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.493934 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.493935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.494191 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:19.494333 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:19.494432 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 08:48:19.494562 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260809 08:48:19.504081 27172 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:48:19.504549 27172 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:48:19.511356 27172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: Generated new TSK 0
I20260809 08:48:19.511492 27172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:48:19.545089 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:19.646545 27192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:19.646732 27192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:19.646761 27192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:19.647861 27192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:19.647900 27192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:19.647934 27192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:48:19.649397 27192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:19.649616 27192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:19.649850 27192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:19.654582 27201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.654608 27203 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.654618 27200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:19.654810 27192 server_base.cc:1034] running on GCE node
I20260809 08:48:19.655068 27192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:19.655280 27192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:19.656421 27192 hybrid_clock.cc:648] HybridClock initialized: now 1786265299656404 us; error 32 us; skew 500 ppm
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:19.658882 27192 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:48:19.659276 27192 webserver.cc:460] Webserver started at http://127.10.184.65:35977/ using document root <none> and password file <none>
I20260809 08:48:19.659466 27192 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:19.659504 27192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:19.659592 27192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:19.660563 27192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "ae197298550d49d296751057be90acf0"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "3099340261fdc3e4d419d3719e00717b"
server_key_iv: "e4cee6cbed01f7677778c7343fd35a2e"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.660857 27192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "ae197298550d49d296751057be90acf0"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "3099340261fdc3e4d419d3719e00717b"
server_key_iv: "e4cee6cbed01f7677778c7343fd35a2e"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.662092 27192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:48:19.662774 27213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.662949 27192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:48:19.663035 27192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "ae197298550d49d296751057be90acf0"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "3099340261fdc3e4d419d3719e00717b"
server_key_iv: "e4cee6cbed01f7677778c7343fd35a2e"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.663092 27192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:19.682487 27192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:19.684441 27192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:19.684567 27192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:19.684770 27192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:19.685114 27192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:19.685149 27192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.685175 27192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:19.685192 27192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.690738 27192 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:42285
I20260809 08:48:19.690778 27414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:42285 every 8 connection(s)
I20260809 08:48:19.691045 27192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:19.699254 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:19.691688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:32817 (local address 127.10.184.126:44979)
0809 08:48:19.691739 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:19.691742 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:19.691940 (+ 198us) server_negotiation.cc:407] Connection header received
0809 08:48:19.692378 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:19.692379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:19.692388 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:19.692436 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:19.693309 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.693807 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.694718 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.694902 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.694945 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:19.696794 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:19.696796 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:19.696797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:19.696804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:19.698043 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.698394 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.698396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.698396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.698453 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.698661 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.698662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.698662 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.698774 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:19.698829 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:19.699073 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 08:48:19.699164 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:48:19.699385 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:19.691673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44979 (local address 127.10.184.65:32817)
0809 08:48:19.691829 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:19.691862 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:19.692305 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:19.692501 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:19.692505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:19.693057 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:19.693238 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:19.693243 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.693867 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.693868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:19.694636 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:19.694638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.694953 (+ 315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.694954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:19.695044 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:19.695414 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:19.695421 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:19.696707 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:19.698118 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:19.698125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:19.698127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:19.698328 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:19.698505 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:19.698506 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:19.698506 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:19.698606 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:19.698827 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:19.698834 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:19.698960 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 08:48:19.699043 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:48:19.699189 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260809 08:48:19.699824 27415 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44979
I20260809 08:48:19.699916 27415 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:19.700088 27415 heartbeater.cc:511] Master 127.10.184.126:44979 requested a full tablet report, sending...
I20260809 08:48:19.700474 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 27192
I20260809 08:48:19.700573 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:48:19.700626 27098 ts_manager.cc:194] Registered new tserver with Master: ae197298550d49d296751057be90acf0 (127.10.184.65:42285)
I20260809 08:48:19.700771 10977 external_mini_cluster.cc:1428] Setting key 1ab31e284bd7e9cefe33f95bb42a5b51
I20260809 08:48:19.701325 27098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:32817
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:48:19.723167 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:19.801683 27422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:19.801859 27422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:19.801880 27422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:19.802989 27422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:19.803030 27422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:19.803058 27422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:48:19.804451 27422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:19.804658 27422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:19.804859 27422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:19.809629 27430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.809639 27429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.809705 27433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:19.809851 27422 server_base.cc:1034] running on GCE node
I20260809 08:48:19.810077 27422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:19.810304 27422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:19.811434 27422 hybrid_clock.cc:648] HybridClock initialized: now 1786265299811420 us; error 26 us; skew 500 ppm
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:19.813741 27422 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:48:19.814118 27422 webserver.cc:460] Webserver started at http://127.10.184.66:37919/ using document root <none> and password file <none>
I20260809 08:48:19.814323 27422 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:19.814370 27422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:19.814494 27422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:19.815465 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "bdc0ea4d9add41ff9ee177d446e9d393"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "04562d0030d5e494552008d30e4e3a19"
server_key_iv: "06633f83cedecb2c3a9030df9f4b3d02"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.815877 27422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "bdc0ea4d9add41ff9ee177d446e9d393"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "04562d0030d5e494552008d30e4e3a19"
server_key_iv: "06633f83cedecb2c3a9030df9f4b3d02"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.817123 27422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:48:19.817752 27445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.817893 27422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.817987 27422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "bdc0ea4d9add41ff9ee177d446e9d393"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "04562d0030d5e494552008d30e4e3a19"
server_key_iv: "06633f83cedecb2c3a9030df9f4b3d02"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.818044 27422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:19.839334 27422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:19.841032 27422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:19.841171 27422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:19.841373 27422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:19.841643 27422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:19.841676 27422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.841704 27422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:19.841728 27422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.847059 27422 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:42383
I20260809 08:48:19.847114 27624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:42383 every 8 connection(s)
I20260809 08:48:19.847397 27422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:19.855512 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:19.848035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:59029 (local address 127.10.184.126:44979)
0809 08:48:19.848087 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:19.848089 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:19.848259 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:48:19.848746 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:19.848746 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:19.848756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:19.848806 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:19.849682 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.850180 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.851119 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.851297 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.851336 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:19.853256 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:19.853258 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:19.853259 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:19.853266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:19.854493 (+ 1227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.854842 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.854842 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.854843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.854888 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:19.855058 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:19.855059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:19.855059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:19.855159 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:19.855203 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:19.855395 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 08:48:19.855450 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:48:19.855692 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:19.847997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44979 (local address 127.10.184.66:59029)
0809 08:48:19.848149 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:19.848189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:19.848670 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:19.848857 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:19.848860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:19.849425 (+ 565us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:19.849617 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:19.849622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.850251 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.850252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:19.851060 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:19.851062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:19.851346 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:19.851347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:19.851464 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:19.851890 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:19.851899 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:19.853176 (+ 1277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:19.854571 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:19.854577 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:19.854578 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:19.854786 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:19.854931 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:19.854932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:19.854932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:19.855017 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:19.855198 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:19.855201 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:19.855302 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 08:48:19.855375 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:48:19.855487 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":85,"threads_started":1}
I20260809 08:48:19.856115 27626 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44979
I20260809 08:48:19.856200 27626 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:19.856357 27626 heartbeater.cc:511] Master 127.10.184.126:44979 requested a full tablet report, sending...
I20260809 08:48:19.856690 27098 ts_manager.cc:194] Registered new tserver with Master: bdc0ea4d9add41ff9ee177d446e9d393 (127.10.184.66:42383)
I20260809 08:48:19.857076 27098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:59029
I20260809 08:48:19.857443 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 27422
I20260809 08:48:19.857543 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:48:19.857722 10977 external_mini_cluster.cc:1428] Setting key 2e7c072a1affcebe7f0a22f924641033
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:48:19.881582 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:48:19.961422 27631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:48:19.961611 27631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:48:19.961632 27631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:48:19.962759 27631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:48:19.962800 27631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:48:19.962826 27631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:48:19.964247 27631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:44979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:48:19.964460 27631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:48:19.964659 27631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:48:19.969486 27637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.969516 27639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:48:19.969517 27641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:48:19.969758 27631 server_base.cc:1034] running on GCE node
I20260809 08:48:19.969914 27631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:48:19.970103 27631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:48:19.971230 27631 hybrid_clock.cc:648] HybridClock initialized: now 1786265299971214 us; error 28 us; skew 500 ppm
Aug 09 08:48:19 dist-test-slave-ncp9 krb5kdc[27031](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:19.973536 27631 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:48:19.973928 27631 webserver.cc:460] Webserver started at http://127.10.184.67:43513/ using document root <none> and password file <none>
I20260809 08:48:19.974130 27631 fs_manager.cc:359] Metadata directory not provided
I20260809 08:48:19.974189 27631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:48:19.974311 27631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:48:19.975266 27631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "1ec855597391439da7f20825edac0f4a"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "55eeb6bae45a220d5746e1dacbf30ddc"
server_key_iv: "317ec9eac3f754efff7a0cb0edac271d"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.975590 27631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "1ec855597391439da7f20825edac0f4a"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "55eeb6bae45a220d5746e1dacbf30ddc"
server_key_iv: "317ec9eac3f754efff7a0cb0edac271d"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.976764 27631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:48:19.977473 27649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.977619 27631 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.977703 27631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "1ec855597391439da7f20825edac0f4a"
format_stamp: "Formatted at 2026-08-09 08:48:19 on dist-test-slave-ncp9"
server_key: "55eeb6bae45a220d5746e1dacbf30ddc"
server_key_iv: "317ec9eac3f754efff7a0cb0edac271d"
server_key_version: "encryptionkey@0"
I20260809 08:48:19.977770 27631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:48:19.994582 27631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:48:19.996281 27631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:48:19.996402 27631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:48:19.996582 27631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:48:19.996963 27631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:48:19.997005 27631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:19.997032 27631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:48:19.997056 27631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:48:20.002481 27631 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:33219
I20260809 08:48:20.002534 27845 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:33219 every 8 connection(s)
I20260809 08:48:20.002794 27631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:48:20.006031 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 27631
I20260809 08:48:20.006129 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:48:20.006318 10977 external_mini_cluster.cc:1428] Setting key 7fc49c90ce7008277d6ccbf0e1d927f6
Aug 09 08:48:20 dist-test-slave-ncp9 krb5kdc[27031](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265299, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:48:20.011778 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.003436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:54181 (local address 127.10.184.126:44979)
0809 08:48:20.003477 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.003479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.003663 (+ 184us) server_negotiation.cc:407] Connection header received
0809 08:48:20.004137 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.004138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.004162 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.004212 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:20.005135 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.005625 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.006907 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.007101 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.007161 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.009252 (+ 2091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:20.009254 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:20.009255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:20.009263 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:20.010532 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:20.010878 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:20.010879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:20.010879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:20.010932 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:20.011156 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:20.011157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:20.011158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:20.011269 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:20.011325 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.011603 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.011684 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:48:20.011936 27849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.003403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44979 (local address 127.10.184.67:54181)
0809 08:48:20.003558 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.003590 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.004050 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.004286 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.004290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.004829 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:48:20.005055 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.005060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.005707 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.005708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.006832 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.006836 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.007166 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.007167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.007367 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.007856 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:48:20.007864 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:48:20.009164 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:48:20.010607 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:20.010614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:20.010616 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:20.010814 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:20.010990 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:48:20.010991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:48:20.010992 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:48:20.011102 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:48:20.011324 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:48:20.011330 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:48:20.011483 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.011568 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.011738 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 08:48:20.012324 27846 heartbeater.cc:347] Connected to a master server at 127.10.184.126:44979
I20260809 08:48:20.012408 27846 heartbeater.cc:464] Registering TS with master...
I20260809 08:48:20.012571 27846 heartbeater.cc:511] Master 127.10.184.126:44979 requested a full tablet report, sending...
I20260809 08:48:20.012926 27098 ts_manager.cc:194] Registered new tserver with Master: 1ec855597391439da7f20825edac0f4a (127.10.184.67:33219)
I20260809 08:48:20.013269 27098 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:54181
I20260809 08:48:20.017727 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:48:20.023929 27160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.018863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.10.184.126:44979)
0809 08:48:20.018899 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.018901 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.018988 (+ 87us) server_negotiation.cc:407] Connection header received
0809 08:48:20.019034 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.019035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.019042 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.019086 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:48:20.019569 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.020017 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.020725 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.020930 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.020973 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.021968 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:48:20.021969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:48:20.021970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:48:20.021974 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:48:20.023088 (+ 1114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:20.023374 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:20.023375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:20.023375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:20.023420 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:48:20.023549 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:48:20.023549 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:48:20.023549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:48:20.023681 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:48:20.023750 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.023817 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.023872 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:48:20.025204 27098 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42976:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:48:20.026727 27098 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:48:20.032876 27864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.028924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42383 (local address 127.0.0.1:49418)
0809 08:48:20.029372 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.029377 (+ 5us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.029412 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.029611 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.029614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.029616 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:48:20.029677 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.029679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030974 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.030975 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.031840 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.031843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.032655 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.032656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.032721 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.032731 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.032786 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.032823 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":37,"thread_start_us":101,"threads_started":1}
I20260809 08:48:20.033079 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.029163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.10.184.66:42383)
0809 08:48:20.029317 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.029321 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.029455 (+ 134us) server_negotiation.cc:407] Connection header received
0809 08:48:20.029506 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.029508 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.029523 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.029589 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:48:20.029762 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030873 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.031925 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.032583 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.032656 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.032712 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.032813 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.032857 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 08:48:20.033221 27868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.028888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:42285 (local address 127.0.0.1:51884)
0809 08:48:20.029090 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.029128 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.029201 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.029836 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.029837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.029838 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:48:20.029874 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.029875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030707 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.030708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.032101 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.032102 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.033000 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.033001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.033068 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.033071 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.033118 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.033161 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":60,"threads_started":1}
I20260809 08:48:20.033493 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.029104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51884 (local address 127.10.184.65:42285)
0809 08:48:20.029532 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.029537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.029546 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:48:20.029585 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.029587 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.029601 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.029829 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:48:20.029938 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030615 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.032179 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.032874 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.032959 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.033018 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.033385 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.033432 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":85,"threads_started":1}
I20260809 08:48:20.033723 27521 tablet_service.cc:1467] Processing CreateTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 (DEFAULT_TABLE table=test-table [id=cef80302f981411b9abfc2c6b6604a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:48:20.034053 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.028908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33219 (local address 127.0.0.1:43388)
0809 08:48:20.029295 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.029303 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.029332 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.029681 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.029682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.029684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:48:20.029730 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.029732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030987 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.030989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.032099 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.032102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.033824 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.033825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.033891 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.033895 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.033949 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.033989 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"mutex_wait_us":24,"thread_start_us":73,"threads_started":1}
I20260809 08:48:20.034094 27521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5046ac8e813d4487af88a6c0d64b07d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:20.034111 27306 tablet_service.cc:1467] Processing CreateTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 (DEFAULT_TABLE table=test-table [id=cef80302f981411b9abfc2c6b6604a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:48:20.034344 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.029082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43388 (local address 127.10.184.67:33219)
0809 08:48:20.029349 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.029352 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.029361 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:48:20.029394 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.029396 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.029409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.029486 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:48:20.029786 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.030847 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.033053 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.033735 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.033807 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.033858 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.034226 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.034271 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":71,"threads_started":1}
I20260809 08:48:20.034449 27306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5046ac8e813d4487af88a6c0d64b07d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:20.034958 27730 tablet_service.cc:1467] Processing CreateTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 (DEFAULT_TABLE table=test-table [id=cef80302f981411b9abfc2c6b6604a5b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:48:20.035303 27730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5046ac8e813d4487af88a6c0d64b07d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:48:20.037498 27876 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: Bootstrap starting.
I20260809 08:48:20.037803 27877 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Bootstrap starting.
I20260809 08:48:20.038133 27876 tablet_bootstrap.cc:654] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:20.038405 27878 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Bootstrap starting.
I20260809 08:48:20.038458 27877 tablet_bootstrap.cc:654] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:20.038702 27876 log.cc:824] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:20.038928 27877 log.cc:824] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:20.039139 27878 tablet_bootstrap.cc:654] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Neither blocks nor log segments found. Creating new log.
I20260809 08:48:20.039484 27876 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: No bootstrap required, opened a new log
I20260809 08:48:20.039556 27876 ts_tablet_manager.cc:1397] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:48:20.039690 27877 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: No bootstrap required, opened a new log
I20260809 08:48:20.039687 27878 log.cc:824] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:48:20.039750 27877 ts_tablet_manager.cc:1397] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:48:20.040264 27878 tablet_bootstrap.cc:492] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: No bootstrap required, opened a new log
I20260809 08:48:20.040323 27878 ts_tablet_manager.cc:1397] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:48:20.041450 27876 raft_consensus.cc:348] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.041532 27877 raft_consensus.cc:348] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.041632 27876 raft_consensus.cc:374] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:20.041647 27877 raft_consensus.cc:374] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:20.041668 27876 raft_consensus.cc:729] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae197298550d49d296751057be90acf0, State: Initialized, Role: FOLLOWER
I20260809 08:48:20.041678 27877 raft_consensus.cc:729] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdc0ea4d9add41ff9ee177d446e9d393, State: Initialized, Role: FOLLOWER
I20260809 08:48:20.041782 27876 consensus_queue.cc:260] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.041782 27877 consensus_queue.cc:260] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.041674 27878 raft_consensus.cc:348] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.041860 27878 raft_consensus.cc:374] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:48:20.041904 27878 raft_consensus.cc:729] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ec855597391439da7f20825edac0f4a, State: Initialized, Role: FOLLOWER
I20260809 08:48:20.042054 27876 ts_tablet_manager.cc:1428] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:48:20.042054 27877 ts_tablet_manager.cc:1428] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:48:20.042025 27878 consensus_queue.cc:260] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.042166 27415 heartbeater.cc:503] Master 127.10.184.126:44979 was elected leader, sending a full tablet report...
I20260809 08:48:20.042232 27626 heartbeater.cc:503] Master 127.10.184.126:44979 was elected leader, sending a full tablet report...
I20260809 08:48:20.042296 27878 ts_tablet_manager.cc:1428] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:48:20.042405 27846 heartbeater.cc:503] Master 127.10.184.126:44979 was elected leader, sending a full tablet report...
W20260809 08:48:20.097900 27627 tablet.cc:2369] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:48:20.155230 27886 raft_consensus.cc:482] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:48:20.155375 27886 raft_consensus.cc:504] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.155756 27886 leader_election.cc:288] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1ec855597391439da7f20825edac0f4a (127.10.184.67:33219), bdc0ea4d9add41ff9ee177d446e9d393 (127.10.184.66:42383)
I20260809 08:48:20.158568 27893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.155922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33219 (local address 127.10.184.65:36675)
0809 08:48:20.156152 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.156160 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.156198 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.156337 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.156338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.156340 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:48:20.156410 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.156412 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.157036 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.157037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.157781 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.157783 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.158333 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.158335 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.158401 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.158406 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.158456 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.158495 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":14,"thread_start_us":125,"threads_started":1}
I20260809 08:48:20.158567 27871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.156105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:36675 (local address 127.10.184.67:33219)
0809 08:48:20.156178 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.156180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.156192 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:48:20.156259 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.156260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.156269 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.156331 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:48:20.156468 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.156960 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.157863 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.158263 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.158320 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.158328 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.158462 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.158497 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:48:20.158682 27417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.155921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42383 (local address 127.10.184.65:40341)
0809 08:48:20.155975 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.155989 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 08:48:20.156041 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:48:20.156233 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:48:20.156235 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:48:20.156237 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:48:20.156312 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.156314 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.157004 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.157005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.157854 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:48:20.157857 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.158445 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.158446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:48:20.158526 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.158530 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:48:20.158582 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:48:20.158622 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:48:20.158692 27873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.156003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40341 (local address 127.10.184.66:42383)
0809 08:48:20.156059 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.156061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.156069 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:20.156096 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.156097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.156106 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.156177 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:48:20.156383 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.156918 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.157923 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.158368 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.158436 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.158444 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.158600 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.158636 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:48:20.158982 27550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5046ac8e813d4487af88a6c0d64b07d7" candidate_uuid: "ae197298550d49d296751057be90acf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" is_pre_election: true
I20260809 08:48:20.159124 27550 raft_consensus.cc:2454] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae197298550d49d296751057be90acf0 in term 0.
I20260809 08:48:20.159075 27768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5046ac8e813d4487af88a6c0d64b07d7" candidate_uuid: "ae197298550d49d296751057be90acf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec855597391439da7f20825edac0f4a" is_pre_election: true
I20260809 08:48:20.159246 27768 raft_consensus.cc:2454] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae197298550d49d296751057be90acf0 in term 0.
I20260809 08:48:20.159355 27214 leader_election.cc:302] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae197298550d49d296751057be90acf0, bdc0ea4d9add41ff9ee177d446e9d393; no voters:
I20260809 08:48:20.159482 27886 raft_consensus.cc:2781] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:48:20.159536 27886 raft_consensus.cc:482] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:48:20.159565 27886 raft_consensus.cc:3037] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:20.160246 27886 raft_consensus.cc:504] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.160354 27886 leader_election.cc:288] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [CANDIDATE]: Term 1 election: Requested vote from peers 1ec855597391439da7f20825edac0f4a (127.10.184.67:33219), bdc0ea4d9add41ff9ee177d446e9d393 (127.10.184.66:42383)
I20260809 08:48:20.160498 27768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5046ac8e813d4487af88a6c0d64b07d7" candidate_uuid: "ae197298550d49d296751057be90acf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1ec855597391439da7f20825edac0f4a"
I20260809 08:48:20.160511 27550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5046ac8e813d4487af88a6c0d64b07d7" candidate_uuid: "ae197298550d49d296751057be90acf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdc0ea4d9add41ff9ee177d446e9d393"
I20260809 08:48:20.160574 27550 raft_consensus.cc:3037] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:20.160573 27768 raft_consensus.cc:3037] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:48:20.161300 27768 raft_consensus.cc:2454] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae197298550d49d296751057be90acf0 in term 1.
I20260809 08:48:20.161370 27550 raft_consensus.cc:2454] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae197298550d49d296751057be90acf0 in term 1.
I20260809 08:48:20.161510 27223 leader_election.cc:302] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1ec855597391439da7f20825edac0f4a, ae197298550d49d296751057be90acf0; no voters:
I20260809 08:48:20.161618 27886 raft_consensus.cc:2781] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:48:20.161788 27886 raft_consensus.cc:686] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 1 LEADER]: Becoming Leader. State: Replica: ae197298550d49d296751057be90acf0, State: Running, Role: LEADER
I20260809 08:48:20.161885 27886 consensus_queue.cc:237] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } }
I20260809 08:48:20.162709 27097 catalog_manager.cc:5495] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 reported cstate change: term changed from 0 to 1, leader changed from <none> to ae197298550d49d296751057be90acf0 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "ae197298550d49d296751057be90acf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae197298550d49d296751057be90acf0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42285 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 } health_report { overall_health: UNKNOWN } } }
I20260809 08:48:20.177711 27872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.175524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51892 (local address 127.10.184.65:42285)
0809 08:48:20.175578 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:48:20.175580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:48:20.175588 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:48:20.175672 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:48:20.175673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:48:20.175688 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:48:20.175744 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:48:20.175850 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.176281 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.177049 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:48:20.177227 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:48:20.177276 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:48:20.177412 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:48:20.177507 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:48:20.177576 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:48:20.177614 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:48:20.177633 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:48:20.179207 27550 raft_consensus.cc:1261] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 1 FOLLOWER]: Refusing update from remote peer ae197298550d49d296751057be90acf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:48:20.179207 27768 raft_consensus.cc:1261] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 1 FOLLOWER]: Refusing update from remote peer ae197298550d49d296751057be90acf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:48:20.179536 27886 consensus_queue.cc:1035] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdc0ea4d9add41ff9ee177d446e9d393" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 42383 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:48:20.179700 27894 consensus_queue.cc:1035] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1ec855597391439da7f20825edac0f4a" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:48:20.181788 27899 mvcc.cc:205] Tried to move back new op lower bound from 7316542669531938816 to 7316542669463744512. Current Snapshot: MvccSnapshot[applied={T|T < 7316542669531938816}]
I20260809 08:48:20.182061 27901 mvcc.cc:205] Tried to move back new op lower bound from 7316542669531938816 to 7316542669463744512. Current Snapshot: MvccSnapshot[applied={T|T < 7316542669531938816}]
I20260809 08:48:20.182418 27902 mvcc.cc:205] Tried to move back new op lower bound from 7316542669531938816 to 7316542669463744512. Current Snapshot: MvccSnapshot[applied={T|T < 7316542669531938816}]
I20260809 08:48:20.183984 27097 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:48:20.184080 27097 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42976:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:48:20.184778 27097 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c672146fdcba4b2c980791297dbec26d: Sending DeleteTablet for 3 replicas of tablet 5046ac8e813d4487af88a6c0d64b07d7
I20260809 08:48:20.185071 27306 tablet_service.cc:1514] Processing DeleteTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:48:20 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:51884
I20260809 08:48:20.185111 27521 tablet_service.cc:1514] Processing DeleteTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:48:20 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:49418
I20260809 08:48:20.185176 27730 tablet_service.cc:1514] Processing DeleteTablet for tablet 5046ac8e813d4487af88a6c0d64b07d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:48:20 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:43388
I20260809 08:48:20.185276 27919 tablet_replica.cc:330] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0: stopping tablet replica
I20260809 08:48:20.185365 27919 raft_consensus.cc:2229] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:48:20.185482 27919 raft_consensus.cc:2258] T 5046ac8e813d4487af88a6c0d64b07d7 P ae197298550d49d296751057be90acf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:48:20.185654 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 27192
I20260809 08:48:20.197005 27920 tablet_replica.cc:330] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: stopping tablet replica
I20260809 08:48:20.185494 27922 tablet_replica.cc:330] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: stopping tablet replica
I20260809 08:48:20.197149 27920 raft_consensus.cc:2229] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:48:20.197252 27922 raft_consensus.cc:2229] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:48:20.197275 27920 raft_consensus.cc:2258] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:48:20.197350 27922 raft_consensus.cc:2258] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:48:20.197757 27920 ts_tablet_manager.cc:1905] T 5046ac8e813d4487af88a6c0d64b07d7 P bdc0ea4d9add41ff9ee177d446e9d393: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:48:20.197944 27922 ts_tablet_manager.cc:1905] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:48:20.199081 27077 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:42285 (error 108)
I20260809 08:48:20.199335 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 27422
I20260809 08:48:20.202539 27922 ts_tablet_manager.cc:1918] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:48:20.202612 27922 log.cc:1196] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/5046ac8e813d4487af88a6c0d64b07d7
I20260809 08:48:20.202890 27922 ts_tablet_manager.cc:1939] T 5046ac8e813d4487af88a6c0d64b07d7 P 1ec855597391439da7f20825edac0f4a: Deleting consensus metadata
I20260809 08:48:20.202915 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.202782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:42285 (local address 127.0.0.1:51902)
0809 08:48:20.202838 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.202881 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:42285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:48:20.203025 27077 catalog_manager.cc:4584] TS ae197298550d49d296751057be90acf0 (127.10.184.65:42285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5046ac8e813d4487af88a6c0d64b07d7: Network error: Client connection negotiation failed: client connection to 127.10.184.65:42285: connect: Connection refused (error 111)
I20260809 08:48:20.203318 27076 catalog_manager.cc:4841] TS 1ec855597391439da7f20825edac0f4a (127.10.184.67:33219): tablet 5046ac8e813d4487af88a6c0d64b07d7 (table test-table [id=cef80302f981411b9abfc2c6b6604a5b]) successfully deleted
I20260809 08:48:20.207957 27867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:48:20.207853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:42383 (local address 127.0.0.1:49434)
0809 08:48:20.207895 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:48:20.207925 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.66:42383: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:48:20.207947 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 27631
I20260809 08:48:20.225615 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 27052
2026-08-09T08:48:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2101 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:48:20 dist-test-slave-ncp9 krb5kdc[27930](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:48:20 dist-test-slave-ncp9 krb5kdc[27930](info): set up 2 sockets
Aug 09 08:48:20 dist-test-slave-ncp9 krb5kdc[27930](info): commencing operation
krb5kdc: starting...
W20260809 08:48:21.343395 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:48:21 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265301, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:48:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:48:21Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangeradmin_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangeradmin_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangerlookup_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangerlookup_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
I20260809 08:48:21.492592 10977 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskAKHc0d/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/postgres -l logfile start
2026-08-09 08:48:23.704 UTC [27978] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:48:23.704 UTC [27978] LOG: listening on IPv4 address "127.10.184.84", port 46421
2026-08-09 08:48:23.708 UTC [27978] LOG: listening on Unix socket "/tmp/.s.PGSQL.46421"
2026-08-09 08:48:23.728 UTC [27982] LOG: database system was shut down at 2026-08-09 08:48:22 UTC
2026-08-09 08:48:23.731 UTC [27978] LOG: database system is ready to accept connections
I20260809 08:48:24.764449 10977 mini_postgres.cc:96] Postgres bound to 46421
2026-08-09 08:48:24.773 UTC [27993] FATAL: database "slave" does not exist
127.10.184.84:46421 - accepting connections
I20260809 08:48:24.773962 10977 mini_ranger.cc:161] Starting Ranger...
I20260809 08:48:24.789582 10977 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:48:25.175026 10977 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:48:25.175105 10977 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin
2026-08-09 08:48:25,545 [I] DB FLAVOR :POSTGRES
2026-08-09 08:48:25,546 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:48:25,546 [I] Checking connection..
2026-08-09 08:48:25,546 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:48:25,814 [I] Checking connection passed.
2026-08-09 08:48:25,814 [I] --------- Verifying version history table ---------
2026-08-09 08:48:25,814 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:48:26,054 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:48:26,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:48:26,309 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:48:26,309 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:48:26,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:48:26.557 UTC [28091] WARNING: there is no transaction in progress
2026-08-09 08:48:26,563 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:48:26,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:48:26,799 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:48:26,799 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:48:26,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:48:27,028 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:48:27,282 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-09 08:48:27,512 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:48:27,752 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:48:27,752 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:48:27,992 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:48:27,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:48:28,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:48:28,498 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:48:28,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:48:29.375 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.389 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.403 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.417 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.431 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.448 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.498 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.504 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.516 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.526 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.535 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.547 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.555 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.561 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.566 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:29.571 UTC [28251] WARNING: there is no transaction in progress
2026-08-09 08:48:30,120 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:48:30,120 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:48:30,367 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:48:30,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:48:30,615 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:48:30,615 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:48:30,870 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:48:30,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:48:31,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:48:31,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-09 08:48:31,593 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:48:31,593 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:48:31,832 [I] DB_PATCHES have already been applied
I20260809 08:48:31.837888 10977 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskAKHc0d/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:48:31.837955 10977 mini_ranger.cc:193] Using host: 127.10.184.84
I20260809 08:48:31.841694 10977 mini_ranger.cc:227] Ranger admin URL: http://127.10.184.84:46153
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:48:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:48:33.026695 10977 mini_ranger.cc:160] Time spent starting Ranger: real 8.253s user 0.000s sys 0.010s
Aug 09, 2026 8:48:51 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:48:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:48:51 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:48:51 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:48:52 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:48:53.380370 10977 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangerkms_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/rangerkms_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.184.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.184.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:48:53 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265333, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:48:53.469491 10977 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:48:53.479892 10977 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:48:54.210121 10977 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:48:54.210211 10977 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms
I20260809 08:48:54.210227 10977 mini_ranger_kms.cc:227] Using postgres at 127.10.184.84:46421
2026-08-09 08:48:54,759 [I] DB FLAVOR :POSTGRES
2026-08-09 08:48:54,760 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:48:54,760 [I] Checking connection
2026-08-09 08:48:54,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:48:55,063 [I] connection success
2026-08-09 08:48:55,064 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:48:55,064 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:48:55,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:48:55,399 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:48:55,399 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:48:55,399 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:48:55,400 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:46421/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:48:55,658 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:48:56.116865 10977 mini_ranger_kms.cc:310] Created kms service
I20260809 08:48:56.275224 10977 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:48:56.316138 10977 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:48:56 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265333, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:48:56.720458 10977 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:48:56.720732 10977 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms:/tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:48:56.720772 10977 mini_ranger_kms.cc:240] Using host: 127.10.184.84
I20260809 08:48:56.724963 10977 mini_ranger_kms.cc:276] Ranger KMS PID: 28522
I20260809 08:48:56.725086 10977 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.184.84:52881
Aug 09, 2026 8:48:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:48:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:48:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:48:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:48:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:48:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:48:58 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265338, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:48:59.129317 10977 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.660s user 0.003s sys 0.522s
I20260809 08:48:59.129421 10977 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:48:59.129513 10977 mini_ranger_kms.cc:401] 127.10.184.84:52881/kms/v1/keys
Aug 09 08:48:59 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265338, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
Aug 09 08:48:59 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265338, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
Aug 09 08:49:01 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:49:02.007723 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34851
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:34851
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:52881/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:02.090407 28609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:02.090605 28609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:02.090626 28609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:02.091758 28609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:02.091796 28609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:02.091809 28609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:02.091820 28609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:02.091830 28609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:02.093441 28609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.184.84:52881/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:34851
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:34851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:02.093660 28609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:02.093874 28609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:02.098790 28619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:02.098820 28617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:02.098789 28615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:02.098973 28609 server_base.cc:1034] running on GCE node
I20260809 08:49:02.099355 28609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:02.099676 28609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:02.100857 28609 hybrid_clock.cc:648] HybridClock initialized: now 1786265342100839 us; error 33 us; skew 500 ppm
Aug 09 08:49:02 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265342, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:02.107925 28609 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:02.108450 28609 webserver.cc:460] Webserver started at http://127.10.184.126:34209/ using document root <none> and password file <none>
I20260809 08:49:02.108717 28609 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:02.108779 28609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:02.108939 28609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:02 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265342, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:02.161258 28609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04"
format_stamp: "Formatted at 2026-08-09 08:49:02 on dist-test-slave-ncp9"
server_key: "fcea3900ee7bb5100630e6c4e75be271"
server_key_iv: "0903f8cdf124cb3251f03d0435a12312"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:02.161687 28609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04"
format_stamp: "Formatted at 2026-08-09 08:49:02 on dist-test-slave-ncp9"
server_key: "fcea3900ee7bb5100630e6c4e75be271"
server_key_iv: "0903f8cdf124cb3251f03d0435a12312"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:02.163494 28609 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:49:02.178931 28629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:02.179102 28609 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:02.179188 28609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04"
format_stamp: "Formatted at 2026-08-09 08:49:02 on dist-test-slave-ncp9"
server_key: "fcea3900ee7bb5100630e6c4e75be271"
server_key_iv: "0903f8cdf124cb3251f03d0435a12312"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:02.179271 28609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:02.205628 28609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:02.207537 28609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:02.207695 28609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:02.213246 28609 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:34851
I20260809 08:49:02.213274 28706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:34851 every 8 connection(s)
I20260809 08:49:02.213672 28609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:02.214394 28707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:02.216809 28707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: Bootstrap starting.
I20260809 08:49:02.217494 28707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:02.217774 28707 log.cc:824] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:02.218494 28707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: No bootstrap required, opened a new log
I20260809 08:49:02.220500 28707 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } }
I20260809 08:49:02.220647 28707 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:02.220680 28707 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe4efc1fb53f4b05b7b1793a7ebcfb04, State: Initialized, Role: FOLLOWER
I20260809 08:49:02.220816 28707 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } }
I20260809 08:49:02.220921 28707 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:02.220964 28707 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:02.221012 28707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:02.221560 28707 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } }
I20260809 08:49:02.221654 28707 leader_election.cc:302] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe4efc1fb53f4b05b7b1793a7ebcfb04; no voters:
I20260809 08:49:02.221804 28707 leader_election.cc:288] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:49:02.221861 28710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:02.222048 28707 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:02.222085 28710 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [term 1 LEADER]: Becoming Leader. State: Replica: fe4efc1fb53f4b05b7b1793a7ebcfb04, State: Running, Role: LEADER
I20260809 08:49:02.222251 28710 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } }
I20260809 08:49:02.222565 28710 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe4efc1fb53f4b05b7b1793a7ebcfb04. Latest consensus state: current_term: 1 leader_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } } }
I20260809 08:49:02.222596 28711 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe4efc1fb53f4b05b7b1793a7ebcfb04" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 34851 } } }
I20260809 08:49:02.222764 28711 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:02.222944 28710 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:02.222971 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 28609
I20260809 08:49:02.223080 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:49:02.223286 10977 external_mini_cluster.cc:1428] Setting key d6c0132ac4519f3a2c1acceecd71c85b
I20260809 08:49:02.224324 28707 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:49:02.226831 28727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:02.227771 28727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:49:02 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:02.237347 28727 catalog_manager.cc:1347] Generated new cluster ID: 627d58205a784557b58a3584e4cc0f36
I20260809 08:49:02.237401 28727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:49:02.239259 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:02.223732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51578 (local address 127.10.184.126:34851)
0809 08:49:02.224816 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:02.224826 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:02.224860 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:49:02.226468 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:02.226519 (+ 51us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:02.226756 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:02.226918 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:02.227713 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:02.229625 (+ 1912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:02.230702 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:02.231341 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:02.231447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:02.234454 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:02.234459 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:02.234459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:02.234472 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:02.237214 (+ 2742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:02.237592 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:02.237595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:02.237596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:02.237727 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:02.237931 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:02.237932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:02.237932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:02.238238 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:02.238391 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:02.238933 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 08:49:02.239045 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1037,"thread_start_us":70,"threads_started":1}
I20260809 08:49:02.260980 28727 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:49:02.261709 28727 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:02.268719 28727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: Generated new TSK 0
I20260809 08:49:02.268883 28727 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:49:03.041090 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:52881/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:03.162194 28776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:03.162452 28776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:03.162504 28776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:03.164189 28776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:03.164247 28776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:03.164284 28776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:03.166389 28776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.184.84:52881/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:03.166628 28776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:03.166893 28776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:03.172386 28784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:03.172448 28788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:03.172647 28785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:03.172847 28776 server_base.cc:1034] running on GCE node
I20260809 08:49:03.173048 28776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:03.173267 28776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:03.174418 28776 hybrid_clock.cc:648] HybridClock initialized: now 1786265343174400 us; error 33 us; skew 500 ppm
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:03.177065 28776 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:03.177575 28776 webserver.cc:460] Webserver started at http://127.10.184.65:42819/ using document root <none> and password file <none>
I20260809 08:49:03.177809 28776 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:03.177865 28776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:03.177990 28776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:03.192036 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "76932d4569a843d9a2616fb20378b8e0"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "a6186cf961b696aef0d7b5d03a0b66e6"
server_key_iv: "40cec24b5ad3898a41dc77202c3ac551"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.192589 28776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "76932d4569a843d9a2616fb20378b8e0"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "a6186cf961b696aef0d7b5d03a0b66e6"
server_key_iv: "40cec24b5ad3898a41dc77202c3ac551"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.194218 28776 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:49:03.215806 28800 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.215960 28776 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.000s sys 0.000s
I20260809 08:49:03.216049 28776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "76932d4569a843d9a2616fb20378b8e0"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "a6186cf961b696aef0d7b5d03a0b66e6"
server_key_iv: "40cec24b5ad3898a41dc77202c3ac551"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.216140 28776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:03.267560 28776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:03.269677 28776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:03.269835 28776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:03.270071 28776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:03.270465 28776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:03.270519 28776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.270550 28776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:03.270572 28776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.278250 28776 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:43539
I20260809 08:49:03.278683 28776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:03.280436 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 28776
I20260809 08:49:03.280542 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:49:03.280735 10977 external_mini_cluster.cc:1428] Setting key 8c3246d34b9cbc84dafd9ffa10214ccc
I20260809 08:49:03.285066 28961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:43539 every 8 connection(s)
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:03.304356 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.279404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34851 (local address 127.10.184.65:41391)
0809 08:49:03.279566 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.279616 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.280243 (+ 627us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.293377 (+ 13134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.293382 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.293622 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:03.293994 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.294002 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.296591 (+ 2589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.296592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.297922 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.297925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.298473 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.298474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.298614 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.299215 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:03.299225 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:03.300823 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:03.302778 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.302784 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.302787 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.303067 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.303269 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.303270 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.303270 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.303389 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.303679 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:03.303685 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:03.303855 (+ 170us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.303975 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.304126 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260809 08:49:03.304591 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.279422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:41391 (local address 127.10.184.126:34851)
0809 08:49:03.292991 (+ 13569us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.292993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.293004 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:49:03.293031 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.293032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.293046 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.293120 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:03.295669 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.296484 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.298031 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.298350 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.298408 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.300943 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:03.300944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:03.300945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:03.300952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:03.302689 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.303147 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.303148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.303149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.303210 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.303455 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.303456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.303457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.303589 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:03.303651 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.304394 (+ 743us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.304483 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13543,"thread_start_us":54,"threads_started":1}
I20260809 08:49:03.305433 28962 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34851
I20260809 08:49:03.305523 28962 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:03.305697 28962 heartbeater.cc:511] Master 127.10.184.126:34851 requested a full tablet report, sending...
I20260809 08:49:03.306272 28656 ts_manager.cc:194] Registered new tserver with Master: 76932d4569a843d9a2616fb20378b8e0 (127.10.184.65:43539)
I20260809 08:49:03.307175 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:41391
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:49:03.322062 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:52881/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 08:49:03.442198 28974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:03.442446 28974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:03.442476 28974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:03.444106 28974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:03.444156 28974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:03.444192 28974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:03.446264 28974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.184.84:52881/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:03.446477 28974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:03.446703 28974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:03.452509 28981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:03.452535 28982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:03.452544 28985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:03.452839 28974 server_base.cc:1034] running on GCE node
I20260809 08:49:03.453028 28974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:03.453243 28974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:03.454378 28974 hybrid_clock.cc:648] HybridClock initialized: now 1786265343454366 us; error 31 us; skew 500 ppm
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:03.456849 28974 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:03.457275 28974 webserver.cc:460] Webserver started at http://127.10.184.66:46195/ using document root <none> and password file <none>
I20260809 08:49:03.457465 28974 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:03.457522 28974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:03.457649 28974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:03.473012 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "bb7e03d50a827f651eb5297bfa7fc6a4"
server_key_iv: "63cc482ae7981575614eeb68eb02cb9e"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.473585 28974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "bb7e03d50a827f651eb5297bfa7fc6a4"
server_key_iv: "63cc482ae7981575614eeb68eb02cb9e"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.475569 28974 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:49:03.488332 28996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.488482 28974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:03.488569 28974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "bb7e03d50a827f651eb5297bfa7fc6a4"
server_key_iv: "63cc482ae7981575614eeb68eb02cb9e"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.488649 28974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:03.514140 28974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:03.516507 28974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:03.516665 28974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:03.516863 28974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:03.517267 28974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:03.517320 28974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.517354 28974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:03.517376 28974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.524492 28974 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:44135
I20260809 08:49:03.525135 28974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:49:03.525689 29161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:44135 every 8 connection(s)
I20260809 08:49:03.528707 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 28974
I20260809 08:49:03.528810 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:49:03.529047 10977 external_mini_cluster.cc:1428] Setting key 915429ff20a8554f349f0351d055ec8e
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:03.537163 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.526493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:36667 (local address 127.10.184.126:34851)
0809 08:49:03.526553 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.526555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.526781 (+ 226us) server_negotiation.cc:407] Connection header received
0809 08:49:03.527445 (+ 664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.527447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.527465 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.527535 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:03.528333 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.529123 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.530896 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.531235 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.531280 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.533898 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:03.533900 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:03.533900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:03.533909 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:03.535542 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.536005 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.536006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.536006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.536065 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.536337 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.536338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.536339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.536471 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:03.536525 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.536962 (+ 437us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.537047 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:49:03.537755 29166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.526381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34851 (local address 127.10.184.66:36667)
0809 08:49:03.526625 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.526672 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.527318 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.527622 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.527627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.527865 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:03.528215 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.528223 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.529248 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.529250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.530795 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.530798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.531372 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.531373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.531514 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.532044 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:03.532054 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:03.533793 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:03.535635 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.535641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.535644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.535922 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.536129 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.536130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.536130 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.536268 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.536548 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:03.536554 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:03.536706 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.536807 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.537509 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":86,"threads_started":1}
I20260809 08:49:03.538414 29163 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34851
I20260809 08:49:03.538504 29163 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:03.538678 29163 heartbeater.cc:511] Master 127.10.184.126:34851 requested a full tablet report, sending...
I20260809 08:49:03.539063 28656 ts_manager.cc:194] Registered new tserver with Master: a5f2d0bc8bf9405a9b82b17a164ac81f (127.10.184.66:44135)
I20260809 08:49:03.539542 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:36667
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:49:03.573741 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:52881/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:03.739488 29174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:03.739822 29174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:03.739912 29174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:03.741674 29174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:03.741779 29174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:03.741859 29174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:03.743980 29174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.184.84:52881/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:34851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:03.744405 29174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:03.744711 29174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:03.761567 29184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:03.762027 29186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:03.764955 29174 server_base.cc:1034] running on GCE node
W20260809 08:49:03.765223 29183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:03.765479 29174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:03.765787 29174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:03.766949 29174 hybrid_clock.cc:648] HybridClock initialized: now 1786265343766924 us; error 37 us; skew 500 ppm
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:03.770313 29174 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:03.770826 29174 webserver.cc:460] Webserver started at http://127.10.184.67:39723/ using document root <none> and password file <none>
I20260809 08:49:03.771060 29174 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:03.771111 29174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:03.771235 29174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:03.785925 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "aff9b289a82346d6b13b2399cad98d23"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "922b94d40d50bb5eb7d70154aeb5eea3"
server_key_iv: "8fff3d493c146ed3df934e0eb4a929b7"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.789649 29174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "aff9b289a82346d6b13b2399cad98d23"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "922b94d40d50bb5eb7d70154aeb5eea3"
server_key_iv: "8fff3d493c146ed3df934e0eb4a929b7"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.794044 29174 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 08:49:03.810146 29193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.810352 29174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:03.810474 29174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "aff9b289a82346d6b13b2399cad98d23"
format_stamp: "Formatted at 2026-08-09 08:49:03 on dist-test-slave-ncp9"
server_key: "922b94d40d50bb5eb7d70154aeb5eea3"
server_key_iv: "8fff3d493c146ed3df934e0eb4a929b7"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:03.810577 29174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:03.830874 29174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:03.836627 29174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:03.836781 29174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:03.837087 29174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:03.837479 29174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:03.837533 29174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.837570 29174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:03.837592 29174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:03.844857 29174 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:36191
I20260809 08:49:03.848086 29174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:49:03.854259 29368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:36191 every 8 connection(s)
I20260809 08:49:03.855003 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 29174
I20260809 08:49:03.855113 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:49:03.855327 10977 external_mini_cluster.cc:1428] Setting key b801befe277a91749dfd2b7e849fc489
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:03.864992 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.854935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:52851 (local address 127.10.184.126:34851)
0809 08:49:03.855511 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.855513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.855528 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:49:03.855997 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.855998 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.856010 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.856040 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:03.856855 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.857666 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.859164 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.859467 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.859507 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.861994 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:03.861996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:03.861996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:03.862004 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:03.863530 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.864016 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.864017 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.864018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.864075 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.864348 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.864349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.864350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.864479 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:03.864516 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.864798 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.864878 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545}
I20260809 08:49:03.865175 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.854939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:34851 (local address 127.10.184.67:52851)
0809 08:49:03.855212 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.855254 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.855869 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.856223 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.856227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.856471 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:03.856748 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.856755 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.857779 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.857779 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.859066 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.859070 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.859595 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.859595 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.859733 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.860224 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:03.860235 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:03.861900 (+ 1665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:03.863621 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.863628 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.863630 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.863909 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.864142 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:03.864143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:03.864143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:03.864270 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:03.864553 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:03.864558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:03.864669 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.864751 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.864886 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":80,"threads_started":1}
I20260809 08:49:03.865808 29369 heartbeater.cc:347] Connected to a master server at 127.10.184.126:34851
I20260809 08:49:03.865906 29369 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:03.866104 29369 heartbeater.cc:511] Master 127.10.184.126:34851 requested a full tablet report, sending...
I20260809 08:49:03.866482 28656 ts_manager.cc:194] Registered new tserver with Master: aff9b289a82346d6b13b2399cad98d23 (127.10.184.67:36191)
I20260809 08:49:03.866892 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:49:03.866920 28656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:52851
I20260809 08:49:03.875830 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.868453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51594 (local address 127.10.184.126:34851)
0809 08:49:03.868488 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.868490 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.868759 (+ 269us) server_negotiation.cc:407] Connection header received
0809 08:49:03.868860 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.868861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.868870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.868920 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:03.869709 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.870401 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.871449 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.871716 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.871758 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.873138 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:03.873139 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:03.873140 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:03.873147 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:03.874596 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.874990 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.874991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.874992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.875049 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:03.875260 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:03.875261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:03.875261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:03.875382 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:03.875453 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.875676 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.875749 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:49:03.879266 28656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:49:03.881544 28656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:49:03.894496 29387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.885182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:44135 (local address 127.0.0.1:54778)
0809 08:49:03.886188 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.886196 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.886235 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.886480 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.886483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.886485 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:03.886552 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.886554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.889697 (+ 3143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.889698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.891347 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.891351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.894212 (+ 2861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.894212 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.894286 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.894303 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.894366 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.894420 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":87,"threads_started":1}
I20260809 08:49:03.896195 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.884652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43539 (local address 127.0.0.1:43112)
0809 08:49:03.885896 (+ 1244us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.885936 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.886005 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.887993 (+ 1988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.887994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.887996 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:03.888038 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.888040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.893027 (+ 4987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.893029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.894120 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.894123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.895692 (+ 1569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.895693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.895757 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.895761 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.895803 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.895840 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1215,"thread_start_us":86,"threads_started":1}
I20260809 08:49:03.896853 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.885446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54778 (local address 127.10.184.66:44135)
0809 08:49:03.886144 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.886148 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.886280 (+ 132us) server_negotiation.cc:407] Connection header received
0809 08:49:03.886319 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.886321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.886336 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.886396 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:03.886632 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.889606 (+ 2974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.891467 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.892244 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.892310 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.892393 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.896724 (+ 4331us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.896773 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":76,"threads_started":1}
I20260809 08:49:03.897749 29052 tablet_service.cc:1467] Processing CreateTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d (DEFAULT_TABLE table=test-table [id=772c3071e00e4ee18bbd96a1aa4727f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:03.898169 29052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552b96ee9fca4d218fb6bd586c757f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:03.902493 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.884938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.10.184.65:43539)
0809 08:49:03.887776 (+ 2838us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.887782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.887792 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:49:03.887823 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.887825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.887840 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.887912 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:03.888110 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.892934 (+ 4824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.894819 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.895543 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.895608 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.895659 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.902370 (+ 6711us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.902421 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2801,"thread_start_us":67,"threads_started":1}
I20260809 08:49:03.903296 28840 tablet_service.cc:1467] Processing CreateTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d (DEFAULT_TABLE table=test-table [id=772c3071e00e4ee18bbd96a1aa4727f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:03.903653 28840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552b96ee9fca4d218fb6bd586c757f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:03.904965 29394 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Bootstrap starting.
I20260809 08:49:03.905954 29394 tablet_bootstrap.cc:654] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:03.906286 29394 log.cc:824] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:03.907509 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.889027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:36191 (local address 127.0.0.1:45086)
0809 08:49:03.896599 (+ 7572us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:03.896609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:03.896640 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:03.903225 (+ 6585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:03.903226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:03.903228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:03.903262 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.903263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.904646 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.904648 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.905946 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:03.905950 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.907309 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.907310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:03.907370 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.907374 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:03.907403 (+ 29us) client_negotiation.cc:240] Negotiation successful
0809 08:49:03.907444 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7560,"thread_start_us":75,"threads_started":1}
I20260809 08:49:03.908073 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:03.889138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45086 (local address 127.10.184.67:36191)
0809 08:49:03.902980 (+ 13842us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:03.902985 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:03.902997 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:49:03.903035 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:03.903037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:03.903053 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:03.903136 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:03.903337 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.904545 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.906488 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:03.907213 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:03.907836 (+ 623us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:03.907924 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:03.907948 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 08:49:03.908000 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13787,"thread_start_us":76,"threads_started":1}
I20260809 08:49:03.909032 29271 tablet_service.cc:1467] Processing CreateTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d (DEFAULT_TABLE table=test-table [id=772c3071e00e4ee18bbd96a1aa4727f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:03.909385 29271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 552b96ee9fca4d218fb6bd586c757f9d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:03.913843 29398 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Bootstrap starting.
I20260809 08:49:03.914734 29398 tablet_bootstrap.cc:654] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:03.915197 29398 log.cc:824] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:03.915652 29400 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Bootstrap starting.
I20260809 08:49:03.915891 29394 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: No bootstrap required, opened a new log
I20260809 08:49:03.915970 29394 ts_tablet_manager.cc:1397] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260809 08:49:03.916880 29400 tablet_bootstrap.cc:654] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:03.917289 29400 log.cc:824] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:03.918097 29398 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: No bootstrap required, opened a new log
I20260809 08:49:03.918215 29398 ts_tablet_manager.cc:1397] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:49:03.918357 29394 raft_consensus.cc:348] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.918545 29394 raft_consensus.cc:374] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:03.918586 29394 raft_consensus.cc:729] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5f2d0bc8bf9405a9b82b17a164ac81f, State: Initialized, Role: FOLLOWER
I20260809 08:49:03.918721 29394 consensus_queue.cc:260] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.918987 29394 ts_tablet_manager.cc:1428] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:49:03.919117 29163 heartbeater.cc:503] Master 127.10.184.126:34851 was elected leader, sending a full tablet report...
I20260809 08:49:03.922313 29398 raft_consensus.cc:348] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.922550 29398 raft_consensus.cc:374] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:03.922664 29398 raft_consensus.cc:729] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76932d4569a843d9a2616fb20378b8e0, State: Initialized, Role: FOLLOWER
I20260809 08:49:03.922878 29398 consensus_queue.cc:260] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.923204 29398 ts_tablet_manager.cc:1428] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 08:49:03.923363 28962 heartbeater.cc:503] Master 127.10.184.126:34851 was elected leader, sending a full tablet report...
I20260809 08:49:03.925676 29400 tablet_bootstrap.cc:492] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: No bootstrap required, opened a new log
I20260809 08:49:03.925803 29400 ts_tablet_manager.cc:1397] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.002s
I20260809 08:49:03.927850 29400 raft_consensus.cc:348] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.928084 29400 raft_consensus.cc:374] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:03.928174 29400 raft_consensus.cc:729] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aff9b289a82346d6b13b2399cad98d23, State: Initialized, Role: FOLLOWER
I20260809 08:49:03.928362 29400 consensus_queue.cc:260] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:03.928694 29400 ts_tablet_manager.cc:1428] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:49:03.928851 29369 heartbeater.cc:503] Master 127.10.184.126:34851 was elected leader, sending a full tablet report...
Aug 09 08:49:03 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
W20260809 08:49:04.026587 29165 tablet.cc:2369] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:49:04.035749 28963 tablet.cc:2369] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:49:04.106009 29370 tablet.cc:2369] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:49:04.152160 29406 raft_consensus.cc:482] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:04.152302 29406 raft_consensus.cc:504] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:04.152729 29406 leader_election.cc:288] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5f2d0bc8bf9405a9b82b17a164ac81f (127.10.184.66:44135), 76932d4569a843d9a2616fb20378b8e0 (127.10.184.65:43539)
I20260809 08:49:04.156821 29371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.153160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:44135 (local address 127.10.184.67:47525)
0809 08:49:04.153215 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:04.153226 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:04.153278 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:04.153502 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:04.153503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:04.153505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:04.153581 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:04.153583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.154529 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.154531 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:04.155769 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:04.155772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.156563 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.156564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:04.156655 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:04.156660 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:49:04.156708 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:49:04.156744 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 08:49:04.157110 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.153059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:47525 (local address 127.10.184.66:44135)
0809 08:49:04.153108 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:04.153110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:04.153331 (+ 221us) server_negotiation.cc:407] Connection header received
0809 08:49:04.153362 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:04.153363 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:04.153376 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:04.153446 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:04.153664 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.154428 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.155872 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.156448 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.156518 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:04.156528 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:04.156998 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 08:49:04.157043 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:49:04.157392 29100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552b96ee9fca4d218fb6bd586c757f9d" candidate_uuid: "aff9b289a82346d6b13b2399cad98d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" is_pre_election: true
I20260809 08:49:04.157539 29100 raft_consensus.cc:2454] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff9b289a82346d6b13b2399cad98d23 in term 0.
I20260809 08:49:04.157754 29197 leader_election.cc:302] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5f2d0bc8bf9405a9b82b17a164ac81f, aff9b289a82346d6b13b2399cad98d23; no voters:
I20260809 08:49:04.157903 29406 raft_consensus.cc:2781] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:49:04.157985 29406 raft_consensus.cc:482] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:49:04.158008 29406 raft_consensus.cc:3037] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:04.158991 29406 raft_consensus.cc:504] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:04.159129 29406 leader_election.cc:288] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [CANDIDATE]: Term 1 election: Requested vote from peers a5f2d0bc8bf9405a9b82b17a164ac81f (127.10.184.66:44135), 76932d4569a843d9a2616fb20378b8e0 (127.10.184.65:43539)
I20260809 08:49:04.159327 29100 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552b96ee9fca4d218fb6bd586c757f9d" candidate_uuid: "aff9b289a82346d6b13b2399cad98d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f"
I20260809 08:49:04.159399 29100 raft_consensus.cc:3037] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:04.160244 29100 raft_consensus.cc:2454] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff9b289a82346d6b13b2399cad98d23 in term 1.
I20260809 08:49:04.160476 29197 leader_election.cc:302] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5f2d0bc8bf9405a9b82b17a164ac81f, aff9b289a82346d6b13b2399cad98d23; no voters:
I20260809 08:49:04.160597 29406 raft_consensus.cc:2781] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:04.160763 29406 raft_consensus.cc:686] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 1 LEADER]: Becoming Leader. State: Replica: aff9b289a82346d6b13b2399cad98d23, State: Running, Role: LEADER
I20260809 08:49:04.160848 29406 consensus_queue.cc:237] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } }
I20260809 08:49:04.161710 28656 catalog_manager.cc:5495] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 reported cstate change: term changed from 0 to 1, leader changed from <none> to aff9b289a82346d6b13b2399cad98d23 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "aff9b289a82346d6b13b2399cad98d23" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aff9b289a82346d6b13b2399cad98d23" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36191 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 } health_report { overall_health: UNKNOWN } } }
I20260809 08:49:04.162165 29388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.154016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:46747 (local address 127.10.184.65:43539)
0809 08:49:04.154060 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:04.154062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:04.154641 (+ 579us) server_negotiation.cc:407] Connection header received
0809 08:49:04.158384 (+ 3743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:04.158386 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:04.158402 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:04.158440 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:04.158806 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.159554 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.161168 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.161763 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.161812 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:04.161820 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:04.162041 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 08:49:04.162084 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:49:04.162557 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.153782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43539 (local address 127.10.184.67:46747)
0809 08:49:04.154568 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:04.154579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:04.154670 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:04.158639 (+ 3969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:04.158640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:04.158643 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:04.158722 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:04.158724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.159753 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.159754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:04.161030 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:04.161061 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.161894 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.161895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:04.161964 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:04.161967 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:49:04.162420 (+ 453us) client_negotiation.cc:240] Negotiation successful
0809 08:49:04.162464 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"thread_start_us":91,"threads_started":1}
I20260809 08:49:04.163007 28877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552b96ee9fca4d218fb6bd586c757f9d" candidate_uuid: "aff9b289a82346d6b13b2399cad98d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76932d4569a843d9a2616fb20378b8e0" is_pre_election: true
I20260809 08:49:04.163164 28877 raft_consensus.cc:2454] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aff9b289a82346d6b13b2399cad98d23 in term 0.
I20260809 08:49:04.163338 28893 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "552b96ee9fca4d218fb6bd586c757f9d" candidate_uuid: "aff9b289a82346d6b13b2399cad98d23" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76932d4569a843d9a2616fb20378b8e0"
I20260809 08:49:04.163426 28893 raft_consensus.cc:3037] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:04.164278 28893 raft_consensus.cc:2454] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aff9b289a82346d6b13b2399cad98d23 in term 1.
I20260809 08:49:04.172968 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.169861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45096 (local address 127.10.184.67:36191)
0809 08:49:04.169940 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:04.169942 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:04.169948 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:49:04.169974 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:04.169975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:04.169989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:04.170053 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:49:04.170227 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.170883 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.171917 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:04.172183 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:04.172241 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:04.172431 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:49:04.172552 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:49:04.172633 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:04.172843 (+ 210us) server_negotiation.cc:299] Negotiation successful
0809 08:49:04.172863 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 09 08:49:04 dist-test-slave-ncp9 krb5kdc[27930](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265343, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:04.174979 29100 raft_consensus.cc:1261] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 1 FOLLOWER]: Refusing update from remote peer aff9b289a82346d6b13b2399cad98d23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:04.175258 28893 raft_consensus.cc:1261] T 552b96ee9fca4d218fb6bd586c757f9d P 76932d4569a843d9a2616fb20378b8e0 [term 1 FOLLOWER]: Refusing update from remote peer aff9b289a82346d6b13b2399cad98d23: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:04.175652 29406 consensus_queue.cc:1035] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5f2d0bc8bf9405a9b82b17a164ac81f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 44135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:04.175804 29406 consensus_queue.cc:1035] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76932d4569a843d9a2616fb20378b8e0" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:04.181011 29413 mvcc.cc:205] Tried to move back new op lower bound from 7316542849737367552 to 7316542849683451904. Current Snapshot: MvccSnapshot[applied={T|T < 7316542849737367552}]
I20260809 08:49:04.183357 29415 mvcc.cc:205] Tried to move back new op lower bound from 7316542849737367552 to 7316542849683451904. Current Snapshot: MvccSnapshot[applied={T|T < 7316542849737367552}]
I20260809 08:49:04.184353 29418 mvcc.cc:205] Tried to move back new op lower bound from 7316542849737367552 to 7316542849683451904. Current Snapshot: MvccSnapshot[applied={T|T < 7316542849737367552}]
I20260809 08:49:04.186604 28656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:04.186712 28656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51594:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:04.187466 28656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fe4efc1fb53f4b05b7b1793a7ebcfb04: Sending DeleteTablet for 3 replicas of tablet 552b96ee9fca4d218fb6bd586c757f9d
I20260809 08:49:04.187791 29271 tablet_service.cc:1514] Processing DeleteTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:04 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:45086
I20260809 08:49:04.188324 28840 tablet_service.cc:1514] Processing DeleteTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:04 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:43112
I20260809 08:49:04.188823 29052 tablet_service.cc:1514] Processing DeleteTablet for tablet 552b96ee9fca4d218fb6bd586c757f9d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:04 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:54778
I20260809 08:49:04.189508 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 28776
I20260809 08:49:04.196453 29435 tablet_replica.cc:330] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: stopping tablet replica
I20260809 08:49:04.196570 29435 raft_consensus.cc:2229] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:04.196693 29435 raft_consensus.cc:2258] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:04.197324 29432 tablet_replica.cc:330] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: stopping tablet replica
I20260809 08:49:04.197429 29432 raft_consensus.cc:2229] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:49:04.197552 29432 raft_consensus.cc:2258] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:04.198077 29432 ts_tablet_manager.cc:1905] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:04.200565 29432 ts_tablet_manager.cc:1918] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:04.200655 29432 log.cc:1196] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/552b96ee9fca4d218fb6bd586c757f9d
I20260809 08:49:04.200974 29432 ts_tablet_manager.cc:1939] T 552b96ee9fca4d218fb6bd586c757f9d P aff9b289a82346d6b13b2399cad98d23: Deleting consensus metadata
I20260809 08:49:04.201406 28633 catalog_manager.cc:4841] TS aff9b289a82346d6b13b2399cad98d23 (127.10.184.67:36191): tablet 552b96ee9fca4d218fb6bd586c757f9d (table test-table [id=772c3071e00e4ee18bbd96a1aa4727f3]) successfully deleted
I20260809 08:49:04.202997 29435 ts_tablet_manager.cc:1905] T 552b96ee9fca4d218fb6bd586c757f9d P a5f2d0bc8bf9405a9b82b17a164ac81f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 08:49:04.205513 28635 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:43539 (error 108)
I20260809 08:49:04.205790 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 28974
I20260809 08:49:04.216236 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.216097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43539 (local address 127.0.0.1:43122)
0809 08:49:04.216158 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:04.216206 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:43539: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 08:49:04.216473 28635 catalog_manager.cc:4584] TS 76932d4569a843d9a2616fb20378b8e0 (127.10.184.65:43539): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 552b96ee9fca4d218fb6bd586c757f9d: Network error: Client connection negotiation failed: client connection to 127.10.184.65:43539: connect: Connection refused (error 111)
I20260809 08:49:04.221526 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 29174
I20260809 08:49:04.223383 29389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:04.223284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:44135 (local address 127.0.0.1:54784)
0809 08:49:04.223330 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:04.223354 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.66:44135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:49:04.228703 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 28609
I20260809 08:49:04.245285 10977 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:49:04.600015 10977 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:49:04.600121 10977 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:49:04.943572 10977 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:49:04.943 UTC [27978] LOG: received smart shutdown request
2026-08-09 08:49:04.947 UTC [27978] LOG: background worker "logical replication launcher" (PID 27990) exited with exit code 1
2026-08-09 08:49:04.947 UTC [27983] LOG: shutting down
2026-08-09 08:49:05.081 UTC [27978] LOG: database system is shut down
2026-08-09T08:49:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (44940 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:49:05 dist-test-slave-ncp9 krb5kdc[29454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:49:05 dist-test-slave-ncp9 krb5kdc[29454](info): set up 2 sockets
Aug 09 08:49:05 dist-test-slave-ncp9 krb5kdc[29454](info): commencing operation
krb5kdc: starting...
W20260809 08:49:06.289510 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:49:06 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:49:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:49:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangeradmin_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangeradmin_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangerlookup_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangerlookup_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
I20260809 08:49:06.437116 10977 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskAKHc0d/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/postgres -l logfile start
2026-08-09 08:49:08.638 UTC [29493] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 08:49:08.638 UTC [29493] LOG: listening on IPv4 address "127.10.184.84", port 56871
2026-08-09 08:49:08.641 UTC [29493] LOG: listening on Unix socket "/tmp/.s.PGSQL.56871"
2026-08-09 08:49:08.660 UTC [29497] LOG: database system was shut down at 2026-08-09 08:49:07 UTC
2026-08-09 08:49:08.663 UTC [29493] LOG: database system is ready to accept connections
I20260809 08:49:09.706220 10977 mini_postgres.cc:96] Postgres bound to 56871
2026-08-09 08:49:09.712 UTC [29512] FATAL: database "slave" does not exist
127.10.184.84:56871 - accepting connections
I20260809 08:49:09.713074 10977 mini_ranger.cc:161] Starting Ranger...
I20260809 08:49:09.726837 10977 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 08:49:10.103449 10977 mini_ranger.cc:87] Created ranger Postgres database
I20260809 08:49:10.103520 10977 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin
2026-08-09 08:49:10,366 [I] DB FLAVOR :POSTGRES
2026-08-09 08:49:10,366 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:49:10,367 [I] Checking connection..
2026-08-09 08:49:10,367 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 08:49:10,621 [I] Checking connection passed.
2026-08-09 08:49:10,621 [I] --------- Verifying version history table ---------
2026-08-09 08:49:10,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:49:10,877 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:49:10,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:49:11,131 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 08:49:11,131 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 08:49:11,131 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 08:49:11.353 UTC [29614] WARNING: there is no transaction in progress
2026-08-09 08:49:11,359 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 08:49:11,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 08:49:11,600 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 08:49:11,600 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:49:11,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 08:49:11,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 08:49:12,087 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-ncp9.c.gcp-upstream.internal','N') ;"
2026-08-09 08:49:12,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:49:12,570 [I] Table x_portal_user does not exist in database ranger
2026-08-09 08:49:12,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:49:12,822 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 08:49:12,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:49:13,070 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:49:13,322 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 08:49:13,322 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 08:49:14.227 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.241 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.255 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.268 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.282 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.298 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.358 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.363 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.376 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.384 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.393 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.405 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.413 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.419 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.424 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14.430 UTC [29781] WARNING: there is no transaction in progress
2026-08-09 08:49:14,949 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 08:49:14,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 08:49:15,176 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 08:49:15,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 08:49:15,423 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 08:49:15,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 08:49:15,680 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 08:49:15,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 08:49:15,947 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 08:49:16,187 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-ncp9.c.gcp-upstream.internal';"
2026-08-09 08:49:16,432 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 08:49:16,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 08:49:16,673 [I] DB_PATCHES have already been applied
I20260809 08:49:16.679114 10977 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskAKHc0d/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 08:49:16.679176 10977 mini_ranger.cc:193] Using host: 127.10.184.84
I20260809 08:49:16.682812 10977 mini_ranger.cc:227] Ranger admin URL: http://127.10.184.84:42583
Aug 09, 2026 8:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:49:16 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 8:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 8:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 8:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 8:49:17 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 08:49:17.861517 10977 mini_ranger.cc:160] Time spent starting Ranger: real 8.148s user 0.001s sys 0.008s
Aug 09, 2026 8:49:36 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 8:49:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 8:49:36 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 8:49:36 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 8:49:37 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 08:49:37.905169 10977 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.10.184.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.10.184.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangerkms_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.10.184.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/rangerkms_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.10.184.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.10.184.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.10.184.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/HTTP_127.10.184.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 08:49:37 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265377, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 08:49:37.983652 10977 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 08:49:37.992050 10977 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 08:49:38.678932 10977 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 08:49:38.679026 10977 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms
I20260809 08:49:38.679042 10977 mini_ranger_kms.cc:227] Using postgres at 127.10.184.84:56871
2026-08-09 08:49:38,832 [I] DB FLAVOR :POSTGRES
2026-08-09 08:49:38,832 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 08:49:38,833 [I] Checking connection
2026-08-09 08:49:38,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 08:49:39,062 [I] connection success
2026-08-09 08:49:39,062 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 08:49:39,062 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 08:49:39,062 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 08:49:39,285 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 08:49:39,286 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 08:49:39,286 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 08:49:39,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.10.184.84:56871/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 08:49:39,536 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 08:49:39.938786 10977 mini_ranger_kms.cc:310] Created kms service
I20260809 08:49:40.138351 10977 mini_ranger_kms.cc:326] Created kudu user
I20260809 08:49:40.188130 10977 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 08:49:40 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265377, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:40.561394 10977 mini_ranger_kms.cc:384] Added ranger policy
I20260809 08:49:40.561509 10977 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms:/tmp/dist-test-taskAKHc0d/build/release/bin/postgresql.jar:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskAKHc0d/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 08:49:40.561529 10977 mini_ranger_kms.cc:240] Using host: 127.10.184.84
I20260809 08:49:40.565892 10977 mini_ranger_kms.cc:276] Ranger KMS PID: 30041
I20260809 08:49:40.565956 10977 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.10.184.84:58449
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 8:49:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 08:49:42 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:43.012389 10977 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.029s user 0.007s sys 0.131s
I20260809 08:49:43.012495 10977 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 08:49:43.012543 10977 mini_ranger_kms.cc:401] 127.10.184.84:58449/kms/v1/keys
Aug 09 08:49:43 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
Aug 09 08:49:43 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265382, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.10.184.84@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
Aug 09 08:49:45 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:49:45.677500 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41957
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41957
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:45.757647 30103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:45.757844 30103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:45.757867 30103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:45.759002 30103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:45.759048 30103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:45.759063 30103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:45.759075 30103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:45.759088 30103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:45.760636 30103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
--ranger_kms_url=127.10.184.84:58449/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41957
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:45.760840 30103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:45.761081 30103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:45.765812 30110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:45.765841 30111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:45.765977 30103 server_base.cc:1034] running on GCE node
W20260809 08:49:45.765828 30115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:45.766220 30103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:45.766553 30103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:45.767730 30103 hybrid_clock.cc:648] HybridClock initialized: now 1786265385767708 us; error 36 us; skew 500 ppm
Aug 09 08:49:45 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265385, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:45.770285 30103 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:45.770689 30103 webserver.cc:460] Webserver started at http://127.10.184.126:34131/ using document root <none> and password file <none>
I20260809 08:49:45.770871 30103 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:45.770910 30103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:45.771014 30103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:45 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265385, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:45.826478 30103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "10e70302920148afaf25163e10fae92d"
format_stamp: "Formatted at 2026-08-09 08:49:45 on dist-test-slave-ncp9"
server_key: "3c498853fb6d92c0e669b1fb7328e030"
server_key_iv: "220b1f6b94136f0c728af18d0757e459"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:45.826848 30103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "10e70302920148afaf25163e10fae92d"
format_stamp: "Formatted at 2026-08-09 08:49:45 on dist-test-slave-ncp9"
server_key: "3c498853fb6d92c0e669b1fb7328e030"
server_key_iv: "220b1f6b94136f0c728af18d0757e459"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:45.828179 30103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:49:45.841666 30125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:45.841851 30103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:45.842005 30103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "10e70302920148afaf25163e10fae92d"
format_stamp: "Formatted at 2026-08-09 08:49:45 on dist-test-slave-ncp9"
server_key: "3c498853fb6d92c0e669b1fb7328e030"
server_key_iv: "220b1f6b94136f0c728af18d0757e459"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:45.842142 30103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:45.863379 30103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:45.865183 30103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:45.865316 30103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:45.869611 30103 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:41957
I20260809 08:49:45.869690 30216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:41957 every 8 connection(s)
I20260809 08:49:45.869994 30103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:45.870746 30217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:45.872741 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30103
I20260809 08:49:45.872850 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:49:45.873091 10977 external_mini_cluster.cc:1428] Setting key 1663a279d147b8eacc439bd15902ca1a
I20260809 08:49:45.873754 30217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Bootstrap starting.
I20260809 08:49:45.874321 30217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:45.874580 30217 log.cc:824] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:45.875240 30217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: No bootstrap required, opened a new log
I20260809 08:49:45.876981 30217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:45.877113 30217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:45.877146 30217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e70302920148afaf25163e10fae92d, State: Initialized, Role: FOLLOWER
I20260809 08:49:45.877281 30217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:45.877364 30217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:45.877395 30217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:45.877445 30217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:45.878048 30217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:45.878161 30217 leader_election.cc:302] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10e70302920148afaf25163e10fae92d; no voters:
I20260809 08:49:45.878319 30217 leader_election.cc:288] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:49:45.878360 30224 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:45.878537 30224 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 LEADER]: Becoming Leader. State: Replica: 10e70302920148afaf25163e10fae92d, State: Running, Role: LEADER
I20260809 08:49:45.878587 30217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:45.878628 30224 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:45.879004 30225 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10e70302920148afaf25163e10fae92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } } }
I20260809 08:49:45.879040 30226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10e70302920148afaf25163e10fae92d. Latest consensus state: current_term: 1 leader_uuid: "10e70302920148afaf25163e10fae92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } } }
I20260809 08:49:45.879110 30225 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:45.879189 30226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: This master's current role is: LEADER
Aug 09 08:49:45 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:45.879366 30229 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:45.879819 30229 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:45.880733 30217 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:49:45.886063 30220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:45.873578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.10.184.126:41957)
0809 08:49:45.873764 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:45.873772 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:45.873796 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:49:45.874227 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:45.874234 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:45.874279 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:45.874403 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:45.875105 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:45.875993 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:45.876729 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:45.877179 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:45.877276 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:45.879877 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:45.879882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:45.879882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:45.879894 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:45.883955 (+ 4061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:45.884425 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:45.884429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:45.884430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:45.884584 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:45.884816 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:45.884817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:45.884817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:45.885357 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:45.885544 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:45.885631 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 08:49:45.885797 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260809 08:49:45.886114 30229 catalog_manager.cc:1347] Generated new cluster ID: 588a46d4d50b4d998d41005ec429ccb9
I20260809 08:49:45.886744 30229 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:49:45.899020 30229 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:49:45.899708 30229 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:45.912189 30229 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Generated new TSK 0
I20260809 08:49:45.912367 30229 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:49:46.540807 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:46.658685 30277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:46.658958 30277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:46.658998 30277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:46.660616 30277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:46.660678 30277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:46.660715 30277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:46.662830 30277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:46.663064 30277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:46.663338 30277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:46.668550 30286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:46.668556 30283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:46.668659 30277 server_base.cc:1034] running on GCE node
W20260809 08:49:46.668802 30282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:46.669062 30277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:46.669322 30277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:46.670516 30277 hybrid_clock.cc:648] HybridClock initialized: now 1786265386670486 us; error 43 us; skew 500 ppm
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:46.673557 30277 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:46.674044 30277 webserver.cc:460] Webserver started at http://127.10.184.65:35369/ using document root <none> and password file <none>
I20260809 08:49:46.674271 30277 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:46.674327 30277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:46.674500 30277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:46.689406 30277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "9ce3970cffbb44c8a32b841ea70cf5dd"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "ead3272a1029346ceb132e1ab4ffa50c"
server_key_iv: "c6e976273cd54321d7294ce2bc2cfe05"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.690054 30277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "9ce3970cffbb44c8a32b841ea70cf5dd"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "ead3272a1029346ceb132e1ab4ffa50c"
server_key_iv: "c6e976273cd54321d7294ce2bc2cfe05"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.691818 30277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:46.703768 30296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.703982 30277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:46.704084 30277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "9ce3970cffbb44c8a32b841ea70cf5dd"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "ead3272a1029346ceb132e1ab4ffa50c"
server_key_iv: "c6e976273cd54321d7294ce2bc2cfe05"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.704171 30277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:46.724658 30277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:46.728696 30277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:46.728924 30277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:46.729207 30277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:46.729612 30277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:46.729700 30277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.729794 30277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:46.729861 30277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.736307 30277 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:34733
I20260809 08:49:46.736490 30461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:34733 every 8 connection(s)
I20260809 08:49:46.736775 30277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:46.739297 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30277
I20260809 08:49:46.739409 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:49:46.739624 10977 external_mini_cluster.cc:1428] Setting key c0f90d003a031e46c13904309ed58f26
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:46.761341 30472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:46.745475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.65:50421)
0809 08:49:46.749076 (+ 3601us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:46.749126 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:46.749748 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:46.750172 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:46.750178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:46.750416 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:46.750691 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:46.750698 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.751690 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.751691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:46.752926 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:46.752930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.753476 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.753477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:46.753612 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:46.754194 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:46.754204 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:46.757826 (+ 3622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:46.759745 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:46.759751 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:46.759754 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:46.760022 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:46.760233 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:46.760234 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:46.760234 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:46.760358 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:46.760636 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:46.760642 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:46.760800 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 08:49:46.760933 (+ 133us) client_negotiation.cc:240] Negotiation successful
0809 08:49:46.761094 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3551,"thread_start_us":84,"threads_started":1}
I20260809 08:49:46.761737 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:46.745464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:50421 (local address 127.10.184.126:41957)
0809 08:49:46.749933 (+ 4469us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:46.749936 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:46.749943 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:49:46.749967 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:46.749968 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:46.749980 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:46.750053 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:46.750813 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.751581 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.753030 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.753336 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.753397 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:46.757928 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:46.757930 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:46.757930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:46.757937 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:46.759659 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:46.760104 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:46.760105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:46.760106 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:46.760169 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:46.760423 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:46.760424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:46.760424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:46.760555 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:46.760610 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:46.761555 (+ 945us) server_negotiation.cc:299] Negotiation successful
0809 08:49:46.761639 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4436,"thread_start_us":80,"threads_started":1}
I20260809 08:49:46.762375 30463 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:46.762470 30463 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:46.762652 30463 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:46.763234 30153 ts_manager.cc:194] Registered new tserver with Master: 9ce3970cffbb44c8a32b841ea70cf5dd (127.10.184.65:34733)
I20260809 08:49:46.764130 30153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:50421
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:49:46.773393 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:46.882741 30475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:46.883111 30475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:46.883204 30475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:46.884869 30475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:46.884960 30475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:46.885003 30475 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:46.887116 30475 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:46.887363 30475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:46.887697 30475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:46.894361 30488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:46.894637 30484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:46.894697 30482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:46.895164 30475 server_base.cc:1034] running on GCE node
I20260809 08:49:46.895363 30475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:46.895622 30475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:46.896795 30475 hybrid_clock.cc:648] HybridClock initialized: now 1786265386896771 us; error 37 us; skew 500 ppm
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:46.900141 30475 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:46.900653 30475 webserver.cc:460] Webserver started at http://127.10.184.66:45003/ using document root <none> and password file <none>
I20260809 08:49:46.900979 30475 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:46.901093 30475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:46.901269 30475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:46.916684 30475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "78c495cfa1cb40da83796997cd6335ed"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "e820f01b1d6ff339b598daa9bf982fa4"
server_key_iv: "846f647e37f38931190919e560119198"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.917320 30475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "78c495cfa1cb40da83796997cd6335ed"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "e820f01b1d6ff339b598daa9bf982fa4"
server_key_iv: "846f647e37f38931190919e560119198"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.920569 30475 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 08:49:46.945883 30498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.946952 30475 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260809 08:49:46.947068 30475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "78c495cfa1cb40da83796997cd6335ed"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "e820f01b1d6ff339b598daa9bf982fa4"
server_key_iv: "846f647e37f38931190919e560119198"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:46.947161 30475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:46.963949 30475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:46.966209 30475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:46.966396 30475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:46.966640 30475 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:46.967070 30475 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:46.967170 30475 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.967242 30475 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:46.967298 30475 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:46.974397 30475 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:40913
I20260809 08:49:46.974980 30475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:49:46.977691 30663 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:40913 every 8 connection(s)
I20260809 08:49:46.982594 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30475
I20260809 08:49:46.982702 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:49:46.983287 10977 external_mini_cluster.cc:1428] Setting key c20ada313745d9139fb2f08395b2058e
Aug 09 08:49:46 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265386, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:46.989867 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:46.977159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:48659 (local address 127.10.184.126:41957)
0809 08:49:46.977218 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:46.977220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:46.978043 (+ 823us) server_negotiation.cc:407] Connection header received
0809 08:49:46.978763 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:46.978764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:46.978781 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:46.978811 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:46.979629 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.980443 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.981969 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.982308 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.982369 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:46.985828 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:46.985830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:46.985830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:46.985838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:46.987737 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:46.988279 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:46.988280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:46.988281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:46.988349 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:46.988725 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:46.988726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:46.988726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:46.988877 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:46.988974 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:46.989672 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 08:49:46.989767 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:49:46.990396 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:46.977288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.66:48659)
0809 08:49:46.977913 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:46.977953 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:46.978621 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:46.978997 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:46.979002 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:46.979242 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:46.979517 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:46.979524 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.980553 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.980554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:46.981872 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:46.981875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:46.982453 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:46.982455 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:46.982592 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:46.983200 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:46.983210 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:46.985715 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:46.987840 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:46.987847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:46.987850 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:46.988181 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:46.988436 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:46.988436 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:46.988437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:46.988644 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:46.989004 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:46.989011 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:46.989171 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 08:49:46.989285 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 08:49:46.989463 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":91,"threads_started":1}
I20260809 08:49:46.991109 30664 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:46.991252 30664 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:46.991503 30664 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:46.991928 30153 ts_manager.cc:194] Registered new tserver with Master: 78c495cfa1cb40da83796997cd6335ed (127.10.184.66:40913)
I20260809 08:49:46.992451 30153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:48659
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:49:47.020776 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:47.109381 30675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:47.109594 30675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:47.109625 30675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:47.110759 30675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:47.110801 30675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:47.110842 30675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:47.112274 30675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:47.112486 30675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:47.112706 30675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:47.117854 30683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.117923 30684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.118166 30688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:47.118378 30675 server_base.cc:1034] running on GCE node
I20260809 08:49:47.118568 30675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:47.118793 30675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:47.119936 30675 hybrid_clock.cc:648] HybridClock initialized: now 1786265387119925 us; error 39 us; skew 500 ppm
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:47.123031 30675 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:47.123517 30675 webserver.cc:460] Webserver started at http://127.10.184.67:39263/ using document root <none> and password file <none>
I20260809 08:49:47.123747 30675 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:47.123803 30675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:47.123929 30675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:47.143908 30675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "3023da3673054030a9c811afe3f22917"
format_stamp: "Formatted at 2026-08-09 08:49:47 on dist-test-slave-ncp9"
server_key: "0858258fcdd67933a504e1cc16c354f4"
server_key_iv: "9b94e6c11ad00c03988fec3126e0cf1f"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.144467 30675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "3023da3673054030a9c811afe3f22917"
format_stamp: "Formatted at 2026-08-09 08:49:47 on dist-test-slave-ncp9"
server_key: "0858258fcdd67933a504e1cc16c354f4"
server_key_iv: "9b94e6c11ad00c03988fec3126e0cf1f"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.150000 30675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.000s
I20260809 08:49:47.163726 30701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.163923 30675 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:47.164040 30675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "3023da3673054030a9c811afe3f22917"
format_stamp: "Formatted at 2026-08-09 08:49:47 on dist-test-slave-ncp9"
server_key: "0858258fcdd67933a504e1cc16c354f4"
server_key_iv: "9b94e6c11ad00c03988fec3126e0cf1f"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.164129 30675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:47.185813 30675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:47.187776 30675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:47.187906 30675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:47.188122 30675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:47.188417 30675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:47.188452 30675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.188480 30675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:47.188505 30675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.194164 30675 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:32851
I20260809 08:49:47.194247 30874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:32851 every 8 connection(s)
I20260809 08:49:47.194559 30675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:49:47.201782 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30675
I20260809 08:49:47.201895 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:49:47.202104 10977 external_mini_cluster.cc:1428] Setting key 22720fa5e7fc53198f2ecbe63ce97ede
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:47.218825 30470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.195458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:39921 (local address 127.10.184.126:41957)
0809 08:49:47.195524 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:47.195526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:47.208587 (+ 13061us) server_negotiation.cc:407] Connection header received
0809 08:49:47.209318 (+ 731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:47.209320 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:47.209332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:47.209359 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:47.210245 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.211065 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.212578 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.212914 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.212954 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.215509 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:47.215510 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:47.215511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:47.215519 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:47.217283 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.217750 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.217751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.217752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.217820 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.218111 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.218111 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.218112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.218263 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:47.218342 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:47.218635 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 08:49:47.218728 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 08:49:47.218997 30878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.195313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.67:39921)
0809 08:49:47.208440 (+ 13127us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:47.208494 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:47.209194 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:47.209600 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:47.209605 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:47.209849 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:47.210135 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.210142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.211182 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.211183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.212476 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.212479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.213065 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.213066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.213202 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.213780 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:47.213790 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:47.215385 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:47.217371 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.217377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.217380 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.217670 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.217893 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.217893 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.217894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.218038 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.218334 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:47.218338 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:47.218489 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 08:49:47.218600 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 08:49:47.218753 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13078,"thread_start_us":9062,"threads_started":1}
I20260809 08:49:47.219523 30876 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:47.219632 30876 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.219830 30876 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:47.220258 30153 ts_manager.cc:194] Registered new tserver with Master: 3023da3673054030a9c811afe3f22917 (127.10.184.67:32851)
I20260809 08:49:47.220743 30153 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:39921
I20260809 08:49:47.224395 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:49:47.225131 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30277
I20260809 08:49:47.254256 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30475
I20260809 08:49:47.267601 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30675
I20260809 08:49:47.287732 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30103
I20260809 08:49:47.299548 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41957
--webserver_interface=127.10.184.126
--webserver_port=34131
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41957
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:47.379652 30879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:47.379855 30879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:47.379885 30879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:47.381026 30879 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:47.381072 30879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:47.381095 30879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:47.381114 30879 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:47.381140 30879 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:47.382728 30879 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41957
--ranger_config_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=34131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:47.382962 30879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:47.383169 30879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:47.387936 30888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.387943 30886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.388001 30890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:47.387977 30879 server_base.cc:1034] running on GCE node
I20260809 08:49:47.388391 30879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:47.388669 30879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:47.389815 30879 hybrid_clock.cc:648] HybridClock initialized: now 1786265387389793 us; error 33 us; skew 500 ppm
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:47.392163 30879 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:47.392582 30879 webserver.cc:460] Webserver started at http://127.10.184.126:34131/ using document root <none> and password file <none>
I20260809 08:49:47.392779 30879 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:47.392839 30879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:47.462338 30879 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:47.463024 30902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.463219 30879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:47.463340 30879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "10e70302920148afaf25163e10fae92d"
format_stamp: "Formatted at 2026-08-09 08:49:45 on dist-test-slave-ncp9"
server_key: "3c498853fb6d92c0e669b1fb7328e030"
server_key_iv: "220b1f6b94136f0c728af18d0757e459"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.463817 30879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:47.492941 30879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:47.494885 30879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:47.495038 30879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:47.499951 30879 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:41957
I20260809 08:49:47.499990 30977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:41957 every 8 connection(s)
I20260809 08:49:47.500349 30879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:47.501652 30978 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:49:47.502314 30978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Bootstrap starting.
I20260809 08:49:47.505113 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30879
I20260809 08:49:47.505455 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:34733
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:41957
--webserver_port=35369
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:47.511140 30978 log.cc:824] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:47.513407 30978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:49:47.513721 30978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Bootstrap complete.
I20260809 08:49:47.517020 30978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:47.517362 30978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10e70302920148afaf25163e10fae92d, State: Initialized, Role: FOLLOWER
I20260809 08:49:47.517546 30978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:47.517627 30978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:47.517685 30978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:47.517735 30978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:49:47.519021 30978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:47.519131 30978 leader_election.cc:302] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10e70302920148afaf25163e10fae92d; no voters:
I20260809 08:49:47.519305 30978 leader_election.cc:288] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:49:47.519358 30983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:49:47.519543 30983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [term 2 LEADER]: Becoming Leader. State: Replica: 10e70302920148afaf25163e10fae92d, State: Running, Role: LEADER
I20260809 08:49:47.519546 30978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:47.519714 30983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } }
I20260809 08:49:47.519994 30984 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "10e70302920148afaf25163e10fae92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } } }
I20260809 08:49:47.519990 30989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10e70302920148afaf25163e10fae92d. Latest consensus state: current_term: 2 leader_uuid: "10e70302920148afaf25163e10fae92d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10e70302920148afaf25163e10fae92d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41957 } } }
I20260809 08:49:47.520098 30984 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:47.520112 30989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:47.520394 30991 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:47.520992 30991 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:47.521131 30991 catalog_manager.cc:1259] Loaded cluster ID: 588a46d4d50b4d998d41005ec429ccb9
I20260809 08:49:47.521226 30991 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:49:47.521436 30978 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 08:49:47.522456 30991 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:47.522627 30991 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Loaded TSK: 0
I20260809 08:49:47.522840 30991 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:49:47.595825 30981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:47.596174 30981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:47.596271 30981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:47.597967 30981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:47.598081 30981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:47.598169 30981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:47.600304 30981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:34733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=35369
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:47.600536 30981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:47.600828 30981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:47.606859 31036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.606870 31034 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.606870 31033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:47.607954 30981 server_base.cc:1034] running on GCE node
I20260809 08:49:47.608161 30981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:47.608408 30981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:47.609570 30981 hybrid_clock.cc:648] HybridClock initialized: now 1786265387609536 us; error 51 us; skew 500 ppm
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:47.613336 30981 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:47.613875 30981 webserver.cc:460] Webserver started at http://127.10.184.65:35369/ using document root <none> and password file <none>
I20260809 08:49:47.614132 30981 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:47.614226 30981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:47.636502 30981 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:47.637300 31047 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.637496 30981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:47.637581 30981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "9ce3970cffbb44c8a32b841ea70cf5dd"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "ead3272a1029346ceb132e1ab4ffa50c"
server_key_iv: "c6e976273cd54321d7294ce2bc2cfe05"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.638815 30981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:47.659955 30981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:47.662343 30981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:47.662554 30981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:47.662848 30981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:47.663257 30981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:47.663341 30981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.663404 30981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:47.663448 30981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.670452 30981 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:34733
I20260809 08:49:47.670974 30981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:47.672688 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 30981
I20260809 08:49:47.673055 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:40913
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:41957
--webserver_port=45003
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:47.677115 31236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:34733 every 8 connection(s)
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:47.683876 31240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.671954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.65:47939)
0809 08:49:47.672336 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:47.672381 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:47.673050 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:47.673699 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:47.673703 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:47.673915 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:47.674182 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.674189 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.675934 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.675935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.677142 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.677145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.677986 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.677988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.678112 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.678573 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:47.678582 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:47.680098 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:47.682239 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.682246 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.682248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.682510 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.682779 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.682779 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.682780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.682894 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.683297 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:47.683303 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:47.683436 (+ 133us) client_negotiation.cc:769] Sending connection context
0809 08:49:47.683511 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 08:49:47.683657 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":84,"threads_started":1}
I20260809 08:49:47.684825 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.672350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:47939 (local address 127.10.184.126:41957)
0809 08:49:47.673251 (+ 901us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:47.673260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:47.673292 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:49:47.673430 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:47.673436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:47.673489 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:47.674312 (+ 823us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:47.674413 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.675809 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.677250 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.677873 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.678351 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.680218 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:47.680222 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:47.680222 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:47.680233 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:47.682119 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.682598 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.682600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.682601 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.682707 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.682959 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.682960 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.682960 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.683218 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:47.684382 (+ 1164us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:47.684462 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:49:47.684585 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":847,"thread_start_us":83,"threads_started":1}
I20260809 08:49:47.688429 31237 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:47.688575 31237 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.688809 31237 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:47.689668 31237 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:41957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:49:47.689797 31237 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.689894 31237 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:47.690215 31237 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.690322 31237 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:47.690601 31237 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:49:47.790119 31246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:47.790308 31246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:47.790339 31246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:47.791568 31246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:47.791664 31246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:47.791723 31246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:47.793550 31246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:40913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=45003
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:47.793916 31246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:47.794221 31246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:47.799927 31252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.799959 31253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.800189 31257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:47.800524 31246 server_base.cc:1034] running on GCE node
I20260809 08:49:47.800701 31246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:47.800935 31246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:47.808269 31246 hybrid_clock.cc:648] HybridClock initialized: now 1786265387808213 us; error 51 us; skew 500 ppm
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:47.811589 31246 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:47.812134 31246 webserver.cc:460] Webserver started at http://127.10.184.66:45003/ using document root <none> and password file <none>
I20260809 08:49:47.812403 31246 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:47.812489 31246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:47.833835 31246 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260809 08:49:47.835178 31268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.835400 31246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:47.835497 31246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "78c495cfa1cb40da83796997cd6335ed"
format_stamp: "Formatted at 2026-08-09 08:49:46 on dist-test-slave-ncp9"
server_key: "e820f01b1d6ff339b598daa9bf982fa4"
server_key_iv: "846f647e37f38931190919e560119198"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:47.835978 31246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:47.858906 31246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:47.861292 31246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:47.861526 31246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:47.861794 31246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:47.862193 31246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:47.862289 31246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.862339 31246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:47.862388 31246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:47.870401 31246 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:40913
I20260809 08:49:47.870577 31433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:40913 every 8 connection(s)
I20260809 08:49:47.870916 31246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:49:47.872469 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 31246
I20260809 08:49:47.872778 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.10.184.84:58449/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:32851
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:41957
--webserver_port=39263
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:47.886616 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.871909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:60659 (local address 127.10.184.126:41957)
0809 08:49:47.871952 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:47.871954 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:47.872253 (+ 299us) server_negotiation.cc:407] Connection header received
0809 08:49:47.875178 (+ 2925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:47.875179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:47.875197 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:47.875279 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:47.876089 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.876856 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.878439 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.878763 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.878813 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.882971 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:47.882972 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:47.882973 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:47.882982 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:47.884818 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.885391 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.885393 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.885393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.885464 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:47.885795 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:47.885796 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:47.885797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:47.885957 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:47.886024 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:47.886421 (+ 397us) server_negotiation.cc:299] Negotiation successful
0809 08:49:47.886520 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:49:47.887395 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:47.871642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.66:60659)
0809 08:49:47.872134 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:47.872177 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:47.875040 (+ 2863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:47.875434 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:47.875440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:47.875674 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:47.875952 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.875959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.877036 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.877037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.878306 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:47.878310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:47.878946 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:47.878947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:47.879085 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:47.881062 (+ 1977us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:47.881072 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:47.882817 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:47.884961 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.884968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.884971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.885278 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.885554 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:47.885555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:47.885555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:47.885697 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:47.886067 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:47.886075 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:47.886236 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 08:49:47.886991 (+ 755us) client_negotiation.cc:240] Negotiation successful
0809 08:49:47.887165 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":79,"threads_started":1}
I20260809 08:49:47.888159 31434 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:47.888258 31434 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.888449 31434 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:47.889243 31434 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:41957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:49:47.889324 31434 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.889390 31434 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:47.889786 31434 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:47.889865 31434 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:47.890251 31434 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:49:47.978274 31438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:47.978467 31438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:47.978497 31438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:47.979796 31438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:47.979842 31438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:47.979876 31438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:47.981793 31438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.10.184.84:58449/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:32851
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=39263
--tserver_master_addrs=127.10.184.126:41957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:47.982028 31438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:47.982276 31438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:47.987496 31454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.987639 31449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:47.988158 31448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:47.988642 31438 server_base.cc:1034] running on GCE node
I20260809 08:49:47.988837 31438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:47.989074 31438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:47.990216 31438 hybrid_clock.cc:648] HybridClock initialized: now 1786265387990205 us; error 26 us; skew 500 ppm
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:47.992807 31438 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:47.993265 31438 webserver.cc:460] Webserver started at http://127.10.184.67:39263/ using document root <none> and password file <none>
I20260809 08:49:47.993470 31438 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:47.993530 31438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 08:49:47 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:48.013115 31438 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 08:49:48.014353 31463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:48.014521 31438 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:48.014616 31438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "3023da3673054030a9c811afe3f22917"
format_stamp: "Formatted at 2026-08-09 08:49:47 on dist-test-slave-ncp9"
server_key: "0858258fcdd67933a504e1cc16c354f4"
server_key_iv: "9b94e6c11ad00c03988fec3126e0cf1f"
server_key_version: "kuduclusterkey@0"
I20260809 08:49:48.014964 31438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:48.037148 31438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:48.038959 31438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:48.039089 31438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:48.039288 31438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:48.039623 31438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:48.039661 31438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:48.039690 31438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:48.039713 31438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:48.046391 31438 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:32851
I20260809 08:49:48.046749 31438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:49:48.047240 31631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:32851 every 8 connection(s)
Aug 09 08:49:48 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265387, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:48.056485 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 31438
I20260809 08:49:48.056622 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:48.047601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59633 (local address 127.10.184.126:41957)
0809 08:49:48.047650 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:48.047652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:48.047880 (+ 228us) server_negotiation.cc:407] Connection header received
0809 08:49:48.048368 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:48.048369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:48.048382 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:48.048433 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:48.048966 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.049489 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.050985 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.051309 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.051385 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:48.054152 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:48.054153 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:48.054154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:48.054162 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:48.055400 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:48.055746 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:48.055748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:48.055748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:48.055800 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:48.056008 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:48.056009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:48.056009 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:48.056125 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:48.056187 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:48.056435 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 08:49:48.056528 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:49:48.056744 31634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:48.047602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41957 (local address 127.10.184.67:59633)
0809 08:49:48.047748 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:48.047787 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:48.048272 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:48.048512 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:48.048516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:48.048672 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:48.048854 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:48.048858 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.049601 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.049602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:48.050881 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:48.050885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.051470 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.051471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:48.051574 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:48.052150 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:48.052168 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:48.054059 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:48.055474 (+ 1415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:48.055480 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:48.055482 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:48.055681 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:48.055852 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:48.055853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:48.055853 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:48.055952 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:48.056184 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:48.056190 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:48.056320 (+ 130us) client_negotiation.cc:769] Sending connection context
0809 08:49:48.056404 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:49:48.056538 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":79,"threads_started":1}
I20260809 08:49:48.057423 31632 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41957
I20260809 08:49:48.057529 31632 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:48.057716 31632 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:48.058406 31632 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:41957 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 08:49:48.058477 31632 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:48.058532 31632 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:48.058876 31632 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:48.058950 31632 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
W20260809 08:49:48.059302 31632 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:49:48.063186 31242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:48.057103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58672 (local address 127.10.184.126:41957)
0809 08:49:48.057188 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:48.057190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:48.057195 (+ 5us) server_negotiation.cc:407] Connection header received
0809 08:49:48.057210 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:48.057211 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:48.057219 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:48.057265 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:48.058069 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.058708 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.059650 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:48.059847 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:48.059897 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:48.060954 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:48.060956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:48.060956 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:48.060963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:48.062138 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:48.062446 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:48.062447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:48.062447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:48.062509 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:48.062666 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:48.062667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:48.062667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:48.062773 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:48.062915 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:48.062960 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:49:48.063027 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 08:49:48 dist-test-slave-ncp9 krb5kdc[29454](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786265388, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:48.690811 31237 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:48.690960 31237 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:48.691362 30926 ts_manager.cc:194] Registered new tserver with Master: 9ce3970cffbb44c8a32b841ea70cf5dd (127.10.184.65:34733)
I20260809 08:49:48.692404 30926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:47939
I20260809 08:49:48.890421 31434 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:48.890583 31434 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:48.890906 30926 ts_manager.cc:194] Registered new tserver with Master: 78c495cfa1cb40da83796997cd6335ed (127.10.184.66:40913)
I20260809 08:49:48.891459 30926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:60659
Aug 09 08:49:48 dist-test-slave-ncp9 krb5kdc[29454](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265388, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for HTTP/127.10.184.84@KRBTEST.COM
I20260809 08:49:49.059471 31632 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:49.059625 31632 heartbeater.cc:511] Master 127.10.184.126:41957 requested a full tablet report, sending...
I20260809 08:49:49.059937 30926 ts_manager.cc:194] Registered new tserver with Master: 3023da3673054030a9c811afe3f22917 (127.10.184.67:32851)
I20260809 08:49:49.060467 30926 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:59633
I20260809 08:49:49.069885 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:49:49.081502 31667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.070950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33568 (local address 127.10.184.126:41957)
0809 08:49:49.071067 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.071069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.071219 (+ 150us) server_negotiation.cc:407] Connection header received
0809 08:49:49.071260 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.071261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.071276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.071362 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:49.072068 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.072859 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.073938 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.074252 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.074335 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.078387 (+ 4052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:49.078389 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:49.078389 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:49.078397 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:49.080225 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:49.080616 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:49.080617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:49.080618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:49.080687 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:49.080910 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:49.080911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:49.080912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:49.081060 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:49.081164 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.081301 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.081389 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260809 08:49:49.083690 30926 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33568:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:49:49.086009 30926 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:49:49.099128 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:40913 (local address 127.0.0.1:34732)
0809 08:49:49.090460 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.090471 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:49.090523 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:49.094964 (+ 4441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:49.094965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:49.094967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:49.095387 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.095389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.096606 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.096607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.097805 (+ 1198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.097808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.098829 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.098830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.098920 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.098934 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:49:49.098994 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:49:49.099044 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":981,"mutex_wait_us":362,"thread_start_us":78,"threads_started":1}
I20260809 08:49:49.099538 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.10.184.66:40913)
0809 08:49:49.094733 (+ 5179us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.094738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.094748 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:49:49.094782 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.094784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.094800 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.094880 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:49.095472 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.096509 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.097903 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.098659 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.098713 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.098769 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.099414 (+ 645us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.099462 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5142,"thread_start_us":61,"threads_started":1}
I20260809 08:49:49.100529 31319 tablet_service.cc:1467] Processing CreateTablet for tablet 421a4767145c4a19ba8afb1d2c57705e (DEFAULT_TABLE table=test-table [id=d60116a7005047bfbafaaf2e9b5a2bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:49.100971 31319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a4767145c4a19ba8afb1d2c57705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:49.101073 31675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34733 (local address 127.0.0.1:33700)
0809 08:49:49.092992 (+ 3339us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.093004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:49.093103 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:49.093392 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:49.093395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:49.093397 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:49.093477 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.093479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.094804 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.094805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.097668 (+ 2863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.097671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.100797 (+ 3126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.100798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.100861 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.100865 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:49.100960 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 08:49:49.101001 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3315,"mutex_wait_us":2,"thread_start_us":68,"threads_started":1}
I20260809 08:49:49.101276 31685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.10.184.65:34733)
0809 08:49:49.090392 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.090397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.093072 (+ 2675us) server_negotiation.cc:407] Connection header received
0809 08:49:49.093194 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.093196 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.093215 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.093270 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:49.093562 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.094700 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.099878 (+ 5178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.100625 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.100683 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.100735 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.101159 (+ 424us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.101204 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":62,"threads_started":1}
I20260809 08:49:49.102309 31684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:32851 (local address 127.0.0.1:45258)
0809 08:49:49.089784 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.089825 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:49.089891 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:49.095202 (+ 5311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:49.095203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:49.095204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:49.095268 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.095272 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.097930 (+ 2658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.097931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.099013 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.099016 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.102107 (+ 3091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.102108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.102170 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.102187 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:49:49.102213 (+ 26us) client_negotiation.cc:240] Negotiation successful
0809 08:49:49.102247 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"mutex_wait_us":14,"thread_start_us":92,"threads_started":1}
I20260809 08:49:49.102756 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.089279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45258 (local address 127.10.184.67:32851)
0809 08:49:49.095052 (+ 5773us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.095056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.095066 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:49:49.095096 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.095098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.095112 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.095243 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:49.095379 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.096362 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.099124 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.102014 (+ 2890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.102545 (+ 531us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.102607 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.102645 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.102685 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5737,"thread_start_us":60,"threads_started":1}
I20260809 08:49:49.103754 31094 tablet_service.cc:1467] Processing CreateTablet for tablet 421a4767145c4a19ba8afb1d2c57705e (DEFAULT_TABLE table=test-table [id=d60116a7005047bfbafaaf2e9b5a2bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:49.103883 31533 tablet_service.cc:1467] Processing CreateTablet for tablet 421a4767145c4a19ba8afb1d2c57705e (DEFAULT_TABLE table=test-table [id=d60116a7005047bfbafaaf2e9b5a2bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:49.104095 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a4767145c4a19ba8afb1d2c57705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:49.104152 31533 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 421a4767145c4a19ba8afb1d2c57705e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:49.105451 31686 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Bootstrap starting.
I20260809 08:49:49.106263 31686 tablet_bootstrap.cc:654] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:49.106666 31686 log.cc:824] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:49.107482 31686 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: No bootstrap required, opened a new log
I20260809 08:49:49.107573 31686 ts_tablet_manager.cc:1397] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:49:49.107853 31692 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Bootstrap starting.
I20260809 08:49:49.108047 31691 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: Bootstrap starting.
I20260809 08:49:49.108683 31691 tablet_bootstrap.cc:654] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:49.109309 31692 tablet_bootstrap.cc:654] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:49.109874 31692 log.cc:824] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:49.109999 31686 raft_consensus.cc:348] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.110190 31686 raft_consensus.cc:374] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:49.110229 31686 raft_consensus.cc:729] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78c495cfa1cb40da83796997cd6335ed, State: Initialized, Role: FOLLOWER
I20260809 08:49:49.110239 31691 log.cc:824] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:49.110757 31686 consensus_queue.cc:260] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.111148 31686 ts_tablet_manager.cc:1428] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 08:49:49.111207 31691 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: No bootstrap required, opened a new log
I20260809 08:49:49.111275 31691 ts_tablet_manager.cc:1397] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 08:49:49.111397 31434 heartbeater.cc:503] Master 127.10.184.126:41957 was elected leader, sending a full tablet report...
I20260809 08:49:49.113539 31692 tablet_bootstrap.cc:492] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: No bootstrap required, opened a new log
I20260809 08:49:49.113656 31692 ts_tablet_manager.cc:1397] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 08:49:49.113641 31691 raft_consensus.cc:348] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.113843 31691 raft_consensus.cc:374] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:49.113929 31691 raft_consensus.cc:729] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ce3970cffbb44c8a32b841ea70cf5dd, State: Initialized, Role: FOLLOWER
I20260809 08:49:49.114075 31691 consensus_queue.cc:260] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.114349 31691 ts_tablet_manager.cc:1428] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:49:49.114516 31237 heartbeater.cc:503] Master 127.10.184.126:41957 was elected leader, sending a full tablet report...
I20260809 08:49:49.116408 31692 raft_consensus.cc:348] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.116622 31692 raft_consensus.cc:374] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:49.116685 31692 raft_consensus.cc:729] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3023da3673054030a9c811afe3f22917, State: Initialized, Role: FOLLOWER
I20260809 08:49:49.116811 31692 consensus_queue.cc:260] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.117115 31692 ts_tablet_manager.cc:1428] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 08:49:49.117246 31632 heartbeater.cc:503] Master 127.10.184.126:41957 was elected leader, sending a full tablet report...
I20260809 08:49:49.121579 31697 raft_consensus.cc:482] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:49.121671 31697 raft_consensus.cc:504] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.122004 31697 leader_election.cc:288] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3023da3673054030a9c811afe3f22917 (127.10.184.67:32851), 9ce3970cffbb44c8a32b841ea70cf5dd (127.10.184.65:34733)
W20260809 08:49:49.122750 31436 tablet.cc:2369] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:49:49.126608 31685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.122592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:57273 (local address 127.10.184.65:34733)
0809 08:49:49.122636 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.122638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.123225 (+ 587us) server_negotiation.cc:407] Connection header received
0809 08:49:49.123276 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.123277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.123287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.123342 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:49.123531 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.124244 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.125654 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.126234 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.126281 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.126290 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.126507 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.126543 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:49:49.126748 31677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.122906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:59305 (local address 127.10.184.67:32851)
0809 08:49:49.122947 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.122949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.123498 (+ 549us) server_negotiation.cc:407] Connection header received
0809 08:49:49.123595 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.123596 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.123605 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.123634 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:49.123836 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.124499 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.125843 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.126344 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.126394 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.126401 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.126647 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.126687 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:49:49.127059 31701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.122643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:32851 (local address 127.10.184.66:59305)
0809 08:49:49.123308 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.123316 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:49.123524 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:49.123696 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:49.123697 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:49.123698 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:49.123766 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.123768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.124602 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.124603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.125731 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.125734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.126476 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.126477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.126559 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.126563 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:49.126917 (+ 354us) client_negotiation.cc:240] Negotiation successful
0809 08:49:49.126962 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"thread_start_us":156,"threads_started":1}
I20260809 08:49:49.127270 31700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.122246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34733 (local address 127.10.184.66:57273)
0809 08:49:49.123004 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.123016 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:49.123217 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:49.123390 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:49.123392 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:49.123394 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:49.123461 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.123462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.124344 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.124345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.125555 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:49.125558 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.126365 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.126366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:49.126435 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.126439 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:49.127008 (+ 569us) client_negotiation.cc:240] Negotiation successful
0809 08:49:49.127053 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":209,"threads_started":1}
I20260809 08:49:49.127482 31562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a4767145c4a19ba8afb1d2c57705e" candidate_uuid: "78c495cfa1cb40da83796997cd6335ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3023da3673054030a9c811afe3f22917" is_pre_election: true
I20260809 08:49:49.127635 31562 raft_consensus.cc:2454] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78c495cfa1cb40da83796997cd6335ed in term 0.
I20260809 08:49:49.128010 31273 leader_election.cc:302] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3023da3673054030a9c811afe3f22917, 78c495cfa1cb40da83796997cd6335ed; no voters:
I20260809 08:49:49.128301 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a4767145c4a19ba8afb1d2c57705e" candidate_uuid: "78c495cfa1cb40da83796997cd6335ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" is_pre_election: true
I20260809 08:49:49.128464 31152 raft_consensus.cc:2454] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78c495cfa1cb40da83796997cd6335ed in term 0.
I20260809 08:49:49.128583 31697 raft_consensus.cc:2781] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:49:49.128650 31697 raft_consensus.cc:482] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:49:49.128675 31697 raft_consensus.cc:3037] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:49.129554 31697 raft_consensus.cc:504] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.129698 31697 leader_election.cc:288] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [CANDIDATE]: Term 1 election: Requested vote from peers 3023da3673054030a9c811afe3f22917 (127.10.184.67:32851), 9ce3970cffbb44c8a32b841ea70cf5dd (127.10.184.65:34733)
I20260809 08:49:49.130112 31152 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a4767145c4a19ba8afb1d2c57705e" candidate_uuid: "78c495cfa1cb40da83796997cd6335ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd"
I20260809 08:49:49.130163 31562 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "421a4767145c4a19ba8afb1d2c57705e" candidate_uuid: "78c495cfa1cb40da83796997cd6335ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3023da3673054030a9c811afe3f22917"
I20260809 08:49:49.130220 31152 raft_consensus.cc:3037] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:49.130234 31562 raft_consensus.cc:3037] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:49.131000 31562 raft_consensus.cc:2454] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78c495cfa1cb40da83796997cd6335ed in term 1.
I20260809 08:49:49.131139 31152 raft_consensus.cc:2454] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78c495cfa1cb40da83796997cd6335ed in term 1.
I20260809 08:49:49.131176 31273 leader_election.cc:302] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3023da3673054030a9c811afe3f22917, 78c495cfa1cb40da83796997cd6335ed; no voters:
I20260809 08:49:49.131343 31697 raft_consensus.cc:2781] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:49.131486 31697 raft_consensus.cc:686] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 1 LEADER]: Becoming Leader. State: Replica: 78c495cfa1cb40da83796997cd6335ed, State: Running, Role: LEADER
I20260809 08:49:49.131565 31697 consensus_queue.cc:237] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } }
I20260809 08:49:49.132251 30926 catalog_manager.cc:5495] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed reported cstate change: term changed from 0 to 1, leader changed from <none> to 78c495cfa1cb40da83796997cd6335ed (127.10.184.66). New cstate: current_term: 1 leader_uuid: "78c495cfa1cb40da83796997cd6335ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78c495cfa1cb40da83796997cd6335ed" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 40913 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 } health_report { overall_health: UNKNOWN } } }
I20260809 08:49:49.144697 31682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:49.141523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.10.184.66:40913)
0809 08:49:49.141560 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:49.141562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:49.141572 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:49:49.141602 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:49.141603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:49.141620 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:49.141687 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:49:49.141836 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.142572 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.143757 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:49.144036 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:49.144099 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:49.144319 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:49:49.144457 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:49:49.144540 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:49.144608 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 08:49:49.144629 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:49:49.146649 31562 raft_consensus.cc:1261] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 1 FOLLOWER]: Refusing update from remote peer 78c495cfa1cb40da83796997cd6335ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:49.146658 31152 raft_consensus.cc:1261] T 421a4767145c4a19ba8afb1d2c57705e P 9ce3970cffbb44c8a32b841ea70cf5dd [term 1 FOLLOWER]: Refusing update from remote peer 78c495cfa1cb40da83796997cd6335ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:49.147033 31702 consensus_queue.cc:1035] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "3023da3673054030a9c811afe3f22917" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 32851 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:49.147192 31702 consensus_queue.cc:1035] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ce3970cffbb44c8a32b841ea70cf5dd" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 34733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:49.150758 31708 mvcc.cc:205] Tried to move back new op lower bound from 7316543033941749760 to 7316543033883332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316543033941749760}]
I20260809 08:49:49.152153 31711 mvcc.cc:205] Tried to move back new op lower bound from 7316543033941749760 to 7316543033883332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316543033941749760}]
I20260809 08:49:49.152441 31709 mvcc.cc:205] Tried to move back new op lower bound from 7316543033941749760 to 7316543033883332608. Current Snapshot: MvccSnapshot[applied={T|T < 7316543033941749760}]
I20260809 08:49:49.154815 30925 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:49.154906 30925 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33568:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:49.155712 30925 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10e70302920148afaf25163e10fae92d: Sending DeleteTablet for 3 replicas of tablet 421a4767145c4a19ba8afb1d2c57705e
I20260809 08:49:49.156162 31319 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a4767145c4a19ba8afb1d2c57705e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:49 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:34732
I20260809 08:49:49.156770 31533 tablet_service.cc:1514] Processing DeleteTablet for tablet 421a4767145c4a19ba8afb1d2c57705e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:49 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:45258
I20260809 08:49:49.157415 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30981
I20260809 08:49:49.158032 31725 tablet_replica.cc:330] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: stopping tablet replica
I20260809 08:49:49.158123 31725 raft_consensus.cc:2229] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:49:49.158217 31725 raft_consensus.cc:2258] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:49.163110 31725 ts_tablet_manager.cc:1905] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:49.164624 31727 tablet_replica.cc:330] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: stopping tablet replica
I20260809 08:49:49.164717 31727 raft_consensus.cc:2229] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:49.164801 31727 raft_consensus.cc:2258] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:49.167547 31725 ts_tablet_manager.cc:1918] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:49.167618 31725 log.cc:1196] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/421a4767145c4a19ba8afb1d2c57705e
I20260809 08:49:49.167888 31725 ts_tablet_manager.cc:1939] T 421a4767145c4a19ba8afb1d2c57705e P 78c495cfa1cb40da83796997cd6335ed: Deleting consensus metadata
I20260809 08:49:49.169319 31727 ts_tablet_manager.cc:1905] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:49.170899 31727 ts_tablet_manager.cc:1918] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:49.170972 31727 log.cc:1196] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/421a4767145c4a19ba8afb1d2c57705e
I20260809 08:49:49.171221 31727 ts_tablet_manager.cc:1939] T 421a4767145c4a19ba8afb1d2c57705e P 3023da3673054030a9c811afe3f22917: Deleting consensus metadata
I20260809 08:49:49.173049 30906 catalog_manager.cc:4841] TS 3023da3673054030a9c811afe3f22917 (127.10.184.67:32851): tablet 421a4767145c4a19ba8afb1d2c57705e (table test-table [id=d60116a7005047bfbafaaf2e9b5a2bdf]) successfully deleted
W20260809 08:49:49.174573 30904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:34733 (error 108)
W20260809 08:49:49.175086 31684 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:49:49.174961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:34733 (local address 127.0.0.1:33710)
0809 08:49:49.175012 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:49.175045 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:34733: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 08:49:49.175192 30904 catalog_manager.cc:4584] TS 9ce3970cffbb44c8a32b841ea70cf5dd (127.10.184.65:34733): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 421a4767145c4a19ba8afb1d2c57705e: Network error: Client connection negotiation failed: client connection to 127.10.184.65:34733: connect: Connection reset by peer (error 104)
I20260809 08:49:49.178700 30904 catalog_manager.cc:4841] TS 78c495cfa1cb40da83796997cd6335ed (127.10.184.66:40913): tablet 421a4767145c4a19ba8afb1d2c57705e (table test-table [id=d60116a7005047bfbafaaf2e9b5a2bdf]) successfully deleted
I20260809 08:49:49.181042 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 31246
I20260809 08:49:49.195200 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 31438
I20260809 08:49:49.204507 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 30879
I20260809 08:49:49.215080 10977 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 08:49:49.549225 10977 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 08:49:49.549317 10977 mini_ranger.cc:66] Stopping Ranger...
I20260809 08:49:49.892552 10977 mini_ranger.cc:68] Stopped Ranger
2026-08-09 08:49:49.892 UTC [29493] LOG: received smart shutdown request
2026-08-09 08:49:49.896 UTC [29493] LOG: background worker "logical replication launcher" (PID 29510) exited with exit code 1
2026-08-09 08:49:49.896 UTC [29500] LOG: shutting down
2026-08-09 08:49:50.027 UTC [29493] LOG: database system is shut down
2026-08-09T08:49:50Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (44967 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:49:50 dist-test-slave-ncp9 krb5kdc[31739](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:49:50 dist-test-slave-ncp9 krb5kdc[31739](info): set up 2 sockets
Aug 09 08:49:50 dist-test-slave-ncp9 krb5kdc[31739](info): commencing operation
krb5kdc: starting...
W20260809 08:49:51.268473 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:49:51.376281 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41651
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:51.457461 31760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:51.457654 31760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:51.457693 31760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:51.458801 31760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:51.458839 31760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:51.458858 31760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:51.458874 31760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:51.458891 31760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:51.460377 31760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:51.460618 31760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:51.460812 31760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:51.465368 31777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:51.465355 31770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.465387 31760 server_base.cc:1034] running on GCE node
W20260809 08:49:51.465370 31769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.465762 31760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:51.466032 31760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:51.467175 31760 hybrid_clock.cc:648] HybridClock initialized: now 1786265391467155 us; error 27 us; skew 500 ppm
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:51.469436 31760 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:51.469830 31760 webserver.cc:460] Webserver started at http://127.10.184.126:38429/ using document root <none> and password file <none>
I20260809 08:49:51.470029 31760 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:51.470077 31760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:51.470206 31760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:51.471155 31760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "5c7db6ae4d674850bdec03b36f816a3e"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "c83f9e8829ee1398ed63a4d0e24652df"
server_key_iv: "a69ce60739b8f7df355872ce89357e1e"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.471486 31760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "5c7db6ae4d674850bdec03b36f816a3e"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "c83f9e8829ee1398ed63a4d0e24652df"
server_key_iv: "a69ce60739b8f7df355872ce89357e1e"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.472726 31760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:49:51.473423 31792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:51.473562 31760 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:51.473645 31760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "5c7db6ae4d674850bdec03b36f816a3e"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "c83f9e8829ee1398ed63a4d0e24652df"
server_key_iv: "a69ce60739b8f7df355872ce89357e1e"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.473711 31760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:51.523176 31760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:51.524962 31760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:51.525096 31760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:51.528769 31760 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:41651
I20260809 08:49:51.528808 31861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:41651 every 8 connection(s)
I20260809 08:49:51.529147 31760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:51.529994 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:51.530494 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 31760
I20260809 08:49:51.530594 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:49:51.530766 10977 external_mini_cluster.cc:1428] Setting key e215b4a203c439b2c7498efac86c78f5
I20260809 08:49:51.531572 31862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.125@KRBTEST.COM" created.
I20260809 08:49:51.540377 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.532229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33236 (local address 127.10.184.126:41651)
0809 08:49:51.532425 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.532435 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.532577 (+ 142us) server_negotiation.cc:407] Connection header received
0809 08:49:51.533129 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.533131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.533148 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.533221 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.533749 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.534545 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.535370 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.535687 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.535790 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.537658 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.537665 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.537665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.537672 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.538982 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.539319 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.539322 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.539323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.539393 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.539583 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.539584 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.539585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.539783 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.539852 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.540021 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.540145 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
I20260809 08:49:51.540381 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.532228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41651 (local address 127.0.0.1:33236)
0809 08:49:51.532457 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.532492 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.532987 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.533238 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.533241 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.533509 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.533660 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.533664 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.534629 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.534630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.535295 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.535297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.535759 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.535759 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.535823 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.536276 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.536284 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.537574 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.539056 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.539063 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.539065 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.539250 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.539451 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.539452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.539452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.539530 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.539844 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.539849 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.539909 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.539989 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.540209 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":84,"spinlock_wait_cycles":7552,"thread_start_us":81,"threads_started":1}
I20260809 08:49:51.541049 31862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.541343 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.541245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52820)
0809 08:49:51.541277 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.541320 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 08:49:51.541465 31797 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
I20260809 08:49:51.541882 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.541790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52836)
0809 08:49:51.541832 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.541860 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 08:49:51.541978 31862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:33829: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
I20260809 08:49:51.555517 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:33829
--webserver_interface=127.10.184.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:51.591215 31862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } attempt: 1
I20260809 08:49:51.591655 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.591534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52848)
0809 08:49:51.591590 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.591621 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 08:49:51.592021 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.591938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52860)
0809 08:49:51.591979 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.592004 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 08:49:51.592110 31862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:33829: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
W20260809 08:49:51.634932 31872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:51.635113 31872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:51.635133 31872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:51.636178 31872 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:51.636210 31872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:51.636222 31872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:51.636233 31872 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:51.636246 31872 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:51.637653 31872 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:33829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.184.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:51.637830 31872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:51.637989 31872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:51.642472 31878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:51.642458 31879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.642489 31872 server_base.cc:1034] running on GCE node
W20260809 08:49:51.642441 31881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.642894 31872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:51.643137 31872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:51.644322 31872 hybrid_clock.cc:648] HybridClock initialized: now 1786265391644291 us; error 28 us; skew 500 ppm
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:51.646692 31872 init.cc:377] Logged in from keytab as kudu/127.10.184.125@KRBTEST.COM (short username kudu)
I20260809 08:49:51.647095 31872 webserver.cc:460] Webserver started at http://127.10.184.125:41977/ using document root <none> and password file <none>
I20260809 08:49:51.647277 31872 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:51.647315 31872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:51.647401 31872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:51.648317 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data/instance:
uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "13eb5fccd1bccf3568334ffb33599b1d"
server_key_iv: "d2d9a820b8014e0eea59c099dc683b84"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.648602 31872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal/instance:
uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "13eb5fccd1bccf3568334ffb33599b1d"
server_key_iv: "d2d9a820b8014e0eea59c099dc683b84"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.649816 31872 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:49:51.650413 31889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:51.650569 31872 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:49:51.650645 31872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "13eb5fccd1bccf3568334ffb33599b1d"
server_key_iv: "d2d9a820b8014e0eea59c099dc683b84"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.650697 31872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:51.660264 31862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } attempt: 2
I20260809 08:49:51.660641 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.660515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52868)
0809 08:49:51.660570 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.660607 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:49:51.660982 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.660878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52870)
0809 08:49:51.660928 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.660962 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 08:49:51.661062 31862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:33829: Network error: Client connection negotiation failed: client connection to 127.10.184.125:33829: connect: Connection refused (error 111)
I20260809 08:49:51.661711 31872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:51.663341 31872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:51.663447 31872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:51.667114 31872 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.125:33829
I20260809 08:49:51.667145 31962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.125:33829 every 8 connection(s)
I20260809 08:49:51.667438 31872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
I20260809 08:49:51.668252 31966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:51.669826 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 31872
I20260809 08:49:51.669934 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-1/wal/instance
I20260809 08:49:51.669816 31966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.670120 10977 external_mini_cluster.cc:1428] Setting key 39c175e6fb96e51f421965d11973b137
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.124@KRBTEST.COM; defaulting to no policy
I20260809 08:49:51.678565 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.670575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33242 (local address 127.10.184.126:41651)
0809 08:49:51.670629 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.670631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.670849 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:49:51.671410 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.671411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.671429 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.671499 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.672202 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.672696 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.673697 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.673911 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.673968 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.675914 (+ 1946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.675915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.675916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.675923 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.677291 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.677655 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.677657 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.677657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.677723 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.677936 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.677937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.677937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.678047 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.678102 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.678357 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.678477 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:49:51.678673 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.670558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41651 (local address 127.0.0.1:33242)
0809 08:49:51.670718 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.670766 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.671317 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.671569 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.671572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.671906 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.672109 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.672114 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.672779 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.672780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.673611 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.673613 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.673981 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.673982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.674099 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.674535 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.674543 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.675830 (+ 1287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.677382 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.677389 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.677391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.677590 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.677776 (+ 186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.677776 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.677777 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.677880 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.678102 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.678109 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.678243 (+ 134us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.678326 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.678468 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":78,"threads_started":1}
I20260809 08:49:51.679175 31966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.124@KRBTEST.COM" created.
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:49:51.685985 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.679380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52872)
0809 08:49:51.679419 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.679429 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.679474 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.679713 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.679714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.679816 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.679873 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.679875 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.680980 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.680981 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.681456 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.681457 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.681894 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.681894 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.681954 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.682310 (+ 356us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.682313 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.683330 (+ 1017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.685147 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.685148 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.685148 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.685296 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.685492 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.685492 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.685492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.685535 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.685792 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.685793 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.685826 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.685870 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.685928 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:49:51.686106 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.679441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52872 (local address 127.10.184.125:33829)
0809 08:49:51.679594 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.679597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.679613 (+ 16us) server_negotiation.cc:407] Connection header received
0809 08:49:51.679627 (+ 14us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.679628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.679638 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.679687 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.679950 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.680841 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.681533 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.681830 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.681888 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.683411 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.683420 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.683420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.683427 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.685076 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.685369 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.685372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.685373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.685436 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.685571 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.685571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.685572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.685733 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.685816 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.685906 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.686009 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":130,"thread_start_us":95,"threads_started":1}
I20260809 08:49:51.686702 31966 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.687008 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.686881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34728)
0809 08:49:51.686926 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.686985 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 08:49:51.687140 31893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
I20260809 08:49:51.687507 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.687417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34730)
0809 08:49:51.687463 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.687490 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:49:51.687613 31966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:37723: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
I20260809 08:49:51.693552 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:37723
--webserver_interface=127.10.184.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:51.736819 31966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } attempt: 1
I20260809 08:49:51.737306 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.737158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34746)
0809 08:49:51.737218 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.737267 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:49:51.737614 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.737534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34750)
0809 08:49:51.737573 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.737598 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 08:49:51.737707 31966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:37723: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
I20260809 08:49:51.752199 31862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } attempt: 3
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:49:51.758903 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.752501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52876)
0809 08:49:51.752559 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.752569 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.752616 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.752765 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.752767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.752985 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.753071 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.753074 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.753733 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.753735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.754302 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.754305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.754662 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.754671 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.754740 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.755148 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.755153 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.756498 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.757980 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.757981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.757981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.758144 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.758324 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.758324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.758325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.758383 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.758598 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.758599 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.758648 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.758707 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.758807 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:49:51.758899 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.752564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52876 (local address 127.10.184.125:33829)
0809 08:49:51.752623 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.752625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.752636 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:49:51.752664 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.752665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.752677 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.752731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.753155 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.753651 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.754380 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.754600 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.754656 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.756577 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.756579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.756579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.756586 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.757902 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.758215 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.758217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.758217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.758269 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.758433 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.758434 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.758434 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.758543 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.758598 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.758738 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.758808 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:49:51.759243 31862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.759541 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.759449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34762)
0809 08:49:51.759489 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.759515 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260809 08:49:51.759876 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.759795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34774)
0809 08:49:51.759836 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.759859 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:49:51.759972 31862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:37723: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
W20260809 08:49:51.773411 31977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:51.773586 31977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:51.773607 31977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:51.774695 31977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:51.774734 31977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:51.774747 31977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:51.774770 31977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:51.774786 31977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:51.776322 31977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41651,127.10.184.125:33829,127.10.184.124:37723
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:37723
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.184.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:51.776538 31977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:51.776742 31977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:51.781335 31992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:51.781350 31988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.781378 31977 server_base.cc:1034] running on GCE node
W20260809 08:49:51.781347 31987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:51.781844 31977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:51.782114 31977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:51.783260 31977 hybrid_clock.cc:648] HybridClock initialized: now 1786265391783253 us; error 32 us; skew 500 ppm
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:51.785542 31977 init.cc:377] Logged in from keytab as kudu/127.10.184.124@KRBTEST.COM (short username kudu)
I20260809 08:49:51.785939 31977 webserver.cc:460] Webserver started at http://127.10.184.124:35899/ using document root <none> and password file <none>
I20260809 08:49:51.786105 31977 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:51.786144 31977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:51.786242 31977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:51.787253 31977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data/instance:
uuid: "cb0a90b2641d456095ea6ae1b2a12b06"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "1acfc3dd1face74b36c76770fd8e85d9"
server_key_iv: "944e8ac315c465e6a322ed2d809efe8b"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.787581 31977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal/instance:
uuid: "cb0a90b2641d456095ea6ae1b2a12b06"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "1acfc3dd1face74b36c76770fd8e85d9"
server_key_iv: "944e8ac315c465e6a322ed2d809efe8b"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.788713 31977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:49:51.789388 32003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:51.789533 31977 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:51.789616 31977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "cb0a90b2641d456095ea6ae1b2a12b06"
format_stamp: "Formatted at 2026-08-09 08:49:51 on dist-test-slave-ncp9"
server_key: "1acfc3dd1face74b36c76770fd8e85d9"
server_key_iv: "944e8ac315c465e6a322ed2d809efe8b"
server_key_version: "encryptionkey@0"
I20260809 08:49:51.789678 31977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:51.791106 31862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } attempt: 1
I20260809 08:49:51.791469 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.791346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34780)
0809 08:49:51.791399 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.791436 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:49:51.791776 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.791694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34782)
0809 08:49:51.791731 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.791757 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 08:49:51.791859 31862 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:37723: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
I20260809 08:49:51.805825 31966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } attempt: 2
I20260809 08:49:51.806180 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.806071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34794)
0809 08:49:51.806124 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.806156 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:49:51.806456 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.806392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34804)
0809 08:49:51.806421 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.806442 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 08:49:51.806533 31966 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:37723: Network error: Client connection negotiation failed: client connection to 127.10.184.124:37723: connect: Connection refused (error 111)
I20260809 08:49:51.806676 31977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:51.808631 31977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:51.808745 31977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:51.812305 31977 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.124:37723
I20260809 08:49:51.812355 32075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.124:37723 every 8 connection(s)
I20260809 08:49:51.812641 31977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
I20260809 08:49:51.813483 32076 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:51.814945 32076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.817792 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 31977
I20260809 08:49:51.817895 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786265241374876-10977-0/minicluster-data/master-2/wal/instance
I20260809 08:49:51.818053 10977 external_mini_cluster.cc:1428] Setting key 30e5e9f73586cd611ced4d5ad7a4aff3
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:49:51.823413 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.815551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.10.184.126:41651)
0809 08:49:51.815588 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.815590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.815784 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:49:51.816187 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.816188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.816198 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.816246 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.816768 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.817299 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.818320 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.818541 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.818596 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.820741 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.820742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.820743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.820750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.822076 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.822550 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.822550 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.822550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.822596 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.822841 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.822841 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.822842 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.822967 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.823031 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.823254 (+ 223us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.823329 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:49:51.823591 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.815522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41651 (local address 127.0.0.1:33252)
0809 08:49:51.815681 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.815719 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.816118 (+ 399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.816295 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.816298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.816518 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.816707 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.816711 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.817370 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.817371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.818243 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.818246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.818603 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.818604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.818754 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.819223 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.819231 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.820659 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.822154 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.822162 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.822183 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.822479 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.822646 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.822647 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.822648 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.822782 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.823027 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.823031 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.823135 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.823217 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.823324 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 08:49:51.824113 32076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:49:51.826637 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.818423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34810 (local address 127.10.184.124:37723)
0809 08:49:51.818719 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.818723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.818740 (+ 17us) server_negotiation.cc:407] Connection header received
0809 08:49:51.818758 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.818760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.818778 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.818844 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.819504 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.820250 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.821160 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.821436 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.821487 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.823809 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.823813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.823814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.823820 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.825481 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.825846 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.825848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.825848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.825898 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.826058 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.826059 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.826059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.826179 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.826276 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.826369 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.826442 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":79,"threads_started":1}
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:49:51.830457 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.824403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52886 (local address 127.10.184.125:33829)
0809 08:49:51.824446 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.824448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.824456 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:51.824480 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.824481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.824490 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.824545 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.824818 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.825521 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.826328 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.826594 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.826664 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.828217 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.828218 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.828219 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.828225 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.829515 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.829870 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.829870 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.829871 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.829917 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.830086 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.830087 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.830087 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.830176 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.830218 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.830348 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.830401 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:49:51.830492 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.824324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:33829 (local address 127.0.0.1:52886)
0809 08:49:51.824364 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.824375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.824410 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.824553 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.824554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.824694 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.824747 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.824749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.825581 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.825582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.826249 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.826252 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.826660 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.826660 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.826730 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.827052 (+ 322us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.827057 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.828137 (+ 1080us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.829600 (+ 1463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.829602 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.829602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.829809 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.829973 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.829974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.829974 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.830034 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.830225 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.830226 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.830265 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.830332 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.830412 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 08:49:51.830799 32076 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:51.833577 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.827324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33258 (local address 127.10.184.126:41651)
0809 08:49:51.827363 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.827365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.827373 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:51.827394 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.827394 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.827403 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.827443 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.827872 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.828332 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.828968 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.829156 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.829213 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.831401 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.831402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.831402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.831409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.832573 (+ 1164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.832884 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.832885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.832885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.832976 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.833156 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.833157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.833157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.833275 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.833432 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.833466 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.833522 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:49:51.836809 32080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.831000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34826)
0809 08:49:51.831076 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.831085 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.831128 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.831242 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.831243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.831365 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.831398 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.831399 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.832182 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.832183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.832628 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.832630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.832976 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.832977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.833022 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.833395 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.833400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.834511 (+ 1111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.835847 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.835848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.835848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.836042 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.836240 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.836241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.836241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.836291 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.836513 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.836514 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.836557 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.836619 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.836695 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 08:49:51.836870 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.831042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34826 (local address 127.10.184.124:37723)
0809 08:49:51.831076 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.831078 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.831116 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:49:51.831189 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.831189 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.831197 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.831235 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.831456 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.832091 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.832704 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.832919 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.832979 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.834569 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.834570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.834570 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.834577 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.835777 (+ 1200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.836105 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.836106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.836106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.836171 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.836339 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.836339 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.836340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.836454 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.836503 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.836652 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.836756 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":17}
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:49:51.838347 32076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: Bootstrap starting.
I20260809 08:49:51.839138 32076 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:51.839407 32076 log.cc:824] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:51.840042 32076 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: No bootstrap required, opened a new log
I20260809 08:49:51.840785 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.834466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52896 (local address 127.10.184.125:33829)
0809 08:49:51.834497 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.834499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.834527 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:49:51.834545 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.834546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.834554 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.834598 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.834985 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.835410 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.836080 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.836337 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.836397 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.838511 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.838512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.838513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.838520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.839768 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.840048 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.840049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.840049 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.840096 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.840287 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.840288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.840288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.840416 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.840595 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.840631 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.840704 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:49:51.841747 32076 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.841858 32076 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:51.841879 32076 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb0a90b2641d456095ea6ae1b2a12b06, State: Initialized, Role: FOLLOWER
I20260809 08:49:51.841995 32076 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.842308 32076 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:51.842428 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:49:51.842265 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.843784 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:49:51.844425 32105 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:49:51.844527 32105 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:49:51.844588 31977 master_runner.cc:186] Error getting master registration for 127.10.184.126:41651: OK, instance_id { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" instance_seqno: 1786265391527764 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:49:51.847218 31977 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:49:51.851889 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.844264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34828 (local address 127.10.184.124:37723)
0809 08:49:51.844295 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.844296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.844712 (+ 416us) server_negotiation.cc:407] Connection header received
0809 08:49:51.844793 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.844794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.844802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.844829 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.845411 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.846072 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.847249 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.847530 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.847583 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.849168 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.849169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.849169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.849176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.850672 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.851053 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.851054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.851054 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.851109 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.851306 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.851306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.851307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.851448 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.851514 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.851721 (+ 207us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.851793 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:49:51.852828 31971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.843853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52912 (local address 127.10.184.125:33829)
0809 08:49:51.843925 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.843926 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.844509 (+ 583us) server_negotiation.cc:407] Connection header received
0809 08:49:51.844603 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.844604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.844613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.844640 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.845340 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.846037 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.847171 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.847532 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.847581 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.849187 (+ 1606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.849188 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.849188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.849194 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.851172 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.851752 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.851753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.851754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.851808 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.852298 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.852299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.852299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.852423 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.852503 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.852687 (+ 184us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.852748 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 08:49:51.854012 31866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.843852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33268 (local address 127.10.184.126:41651)
0809 08:49:51.843890 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.843891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.844018 (+ 127us) server_negotiation.cc:407] Connection header received
0809 08:49:51.844106 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.844107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.844116 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.844143 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.847914 (+ 3771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.848606 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.849593 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.849870 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.849909 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.851180 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.851182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.851182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.851189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.852935 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.853333 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.853335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.853335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.853392 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.853571 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.853571 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.853572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.853657 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.853734 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.853860 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.853925 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:49:51.866972 31862 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } attempt: 2
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:49:51.873103 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.867273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34834 (local address 127.10.184.124:37723)
0809 08:49:51.867326 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.867327 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.867338 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:49:51.867357 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.867357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.867365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.867418 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.867637 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.868254 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.868981 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.869176 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.869227 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.870981 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.870983 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.870983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.870989 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.872129 (+ 1140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.872472 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.872472 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.872473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.872514 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.872674 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.872674 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.872674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.872755 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.872795 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.872996 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.873051 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:49:51.873144 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.867219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34834)
0809 08:49:51.867268 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.867279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.867320 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.867420 (+ 100us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.867421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.867530 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.867558 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.867559 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.868340 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.868341 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.868865 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.868866 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.869233 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.869234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.869284 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.869614 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.869619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.870907 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.872204 (+ 1297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.872205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.872205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.872407 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.872562 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.872562 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.872563 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.872621 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.872802 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.872802 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.872844 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.872966 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.873049 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:49:51.874550 31862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: Bootstrap starting.
I20260809 08:49:51.875129 31862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:51.875425 31862 log.cc:824] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:51.876278 31862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: No bootstrap required, opened a new log
I20260809 08:49:51.878278 31862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.878415 31862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:51.878455 31862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c7db6ae4d674850bdec03b36f816a3e, State: Initialized, Role: FOLLOWER
I20260809 08:49:51.878568 31862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.878818 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.878870 31862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:51.878898 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:49:51.880558 32131 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:49:51.880656 32131 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:49:51.881289 31760 master_runner.cc:186] Error getting master registration for 127.10.184.125:33829: OK, instance_id { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" instance_seqno: 1786265391666086 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:49:51.883486 31760 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:49:51.884212 32084 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:51.884298 32084 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.884538 32084 leader_election.cc:288] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c7db6ae4d674850bdec03b36f816a3e (127.10.184.126:41651), d5ebd9ba4fb34e66b5c9ddf30309e51c (127.10.184.125:33829)
I20260809 08:49:51.885200 31829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7db6ae4d674850bdec03b36f816a3e" is_pre_election: true
I20260809 08:49:51.885335 31829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb0a90b2641d456095ea6ae1b2a12b06 in term 0.
I20260809 08:49:51.885555 32004 leader_election.cc:302] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7db6ae4d674850bdec03b36f816a3e, cb0a90b2641d456095ea6ae1b2a12b06; no voters:
I20260809 08:49:51.885690 32084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:49:51.885737 32084 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:49:51.885761 32084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:51.886615 32084 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.886749 32084 leader_election.cc:288] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [CANDIDATE]: Term 1 election: Requested vote from peers 5c7db6ae4d674850bdec03b36f816a3e (127.10.184.126:41651), d5ebd9ba4fb34e66b5c9ddf30309e51c (127.10.184.125:33829)
I20260809 08:49:51.887141 31829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c7db6ae4d674850bdec03b36f816a3e"
I20260809 08:49:51.887228 31829 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:51.888069 31829 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0a90b2641d456095ea6ae1b2a12b06 in term 1.
I20260809 08:49:51.888494 32004 leader_election.cc:302] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c7db6ae4d674850bdec03b36f816a3e, cb0a90b2641d456095ea6ae1b2a12b06; no voters:
I20260809 08:49:51.888697 32084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:51.888859 32084 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [term 1 LEADER]: Becoming Leader. State: Replica: cb0a90b2641d456095ea6ae1b2a12b06, State: Running, Role: LEADER
I20260809 08:49:51.888962 32084 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.889345 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb0a90b2641d456095ea6ae1b2a12b06. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.889415 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:51.891356 32138 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:51.891556 32138 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:51.893639 31829 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [term 1 FOLLOWER]: Refusing update from remote peer cb0a90b2641d456095ea6ae1b2a12b06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:51.894016 32084 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:51.895378 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb0a90b2641d456095ea6ae1b2a12b06. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.895459 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:49:51.897688 31966 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } attempt: 3
I20260809 08:49:51.901454 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.901538 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:51.901595 32114 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.901659 32114 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:49:51.901808 32139 mvcc.cc:205] Tried to move back new op lower bound from 7316543045192781824 to 7316543045177868288. Current Snapshot: MvccSnapshot[applied={T|T < 7316543045192781824}]
Aug 09 08:49:51 dist-test-slave-ncp9 krb5kdc[31739](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265391, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:49:51.902842 32138 catalog_manager.cc:1347] Generated new cluster ID: cabf377c67994067802fef5bb3b45c79
I20260809 08:49:51.902931 32138 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:49:51.906908 31969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.897953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:37723 (local address 127.0.0.1:34848)
0809 08:49:51.898043 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:51.898056 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:51.898093 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:51.898230 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:51.898231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:51.898407 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:51.898455 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.898457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.899354 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.899355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.900155 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:51.900157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.900619 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.900619 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:51.900677 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.901144 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:51.901157 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:51.902706 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:51.904447 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.904449 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.904449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.904684 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.906225 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:51.906226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:51.906227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:51.906300 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:51.906597 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:51.906598 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:51.906646 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:49:51.906712 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:49:51.906807 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260809 08:49:51.907240 32082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:51.898056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34848 (local address 127.10.184.124:37723)
0809 08:49:51.898096 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:51.898098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:51.898131 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:49:51.898160 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:51.898161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:51.898170 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:51.898220 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:51.898523 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.899261 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.900240 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:51.900546 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:51.900615 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:51.902787 (+ 2172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:51.902788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:51.902788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:51.902795 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:51.904363 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.904762 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.904763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.904764 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.904842 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:51.906380 (+ 1538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:51.906381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:51.906381 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:51.906536 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:51.906605 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:51.907070 (+ 465us) server_negotiation.cc:299] Negotiation successful
0809 08:49:51.907156 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:49:51.908596 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: Bootstrap starting.
I20260809 08:49:51.909292 31966 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:51.909588 31966 log.cc:824] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:51.909875 32140 mvcc.cc:205] Tried to move back new op lower bound from 7316543045192781824 to 7316543045177868288. Current Snapshot: MvccSnapshot[applied={T|T < 7316543045192781824 or (T in {7316543045192781824})}]
I20260809 08:49:51.910647 31966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: No bootstrap required, opened a new log
I20260809 08:49:51.912602 31966 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.912739 31966 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:51.912773 31966 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ebd9ba4fb34e66b5c9ddf30309e51c, State: Initialized, Role: FOLLOWER
I20260809 08:49:51.913087 31966 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } }
I20260809 08:49:51.913786 31926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c"
I20260809 08:49:51.913901 31926 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:51.914710 31926 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb0a90b2641d456095ea6ae1b2a12b06 in term 1.
W20260809 08:49:51.914871 31895 tablet.cc:2369] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:49:51.915269 31925 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 1 FOLLOWER]: Refusing update from remote peer cb0a90b2641d456095ea6ae1b2a12b06: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:51.915529 31927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" is_pre_election: true
I20260809 08:49:51.915599 31927 raft_consensus.cc:2362] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate cb0a90b2641d456095ea6ae1b2a12b06 in term 1. Re-sending same reply.
I20260809 08:49:51.915943 32134 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:51.918617 32152 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.918752 32152 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:49:51.918829 32152 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb0a90b2641d456095ea6ae1b2a12b06. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.918933 32152 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:49:51.919003 32152 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.919093 32152 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:49:51.919184 31966 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:51.919654 32084 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c7db6ae4d674850bdec03b36f816a3e" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41651 } } peers { permanent_uuid: "d5ebd9ba4fb34e66b5c9ddf30309e51c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 33829 } } peers { permanent_uuid: "cb0a90b2641d456095ea6ae1b2a12b06" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 37723 } } }
I20260809 08:49:51.919750 32084 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:51.921159 32138 catalog_manager.cc:1370] Generated new certificate authority record
W20260809 08:49:51.921392 32178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:49:51.921437 32178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:49:51.921837 32138 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:51.928946 32153 mvcc.cc:205] Tried to move back new op lower bound from 7316543045287297024 to 7316543045177868288. Current Snapshot: MvccSnapshot[applied={T|T < 7316543045192781824}]
I20260809 08:49:51.931170 32138 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb0a90b2641d456095ea6ae1b2a12b06: Generated new TSK 0
I20260809 08:49:51.931326 32138 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:49:52.881137 32131 catalog_manager.cc:1259] Loaded cluster ID: cabf377c67994067802fef5bb3b45c79
I20260809 08:49:52.881217 32131 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: loading cluster ID for follower catalog manager: success
I20260809 08:49:52.882023 32131 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: acquiring CA information for follower catalog manager: success
I20260809 08:49:52.882386 32131 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5c7db6ae4d674850bdec03b36f816a3e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:49:52.921836 32178 catalog_manager.cc:1259] Loaded cluster ID: cabf377c67994067802fef5bb3b45c79
I20260809 08:49:52.921916 32178 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: loading cluster ID for follower catalog manager: success
I20260809 08:49:52.922730 32178 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: acquiring CA information for follower catalog manager: success
I20260809 08:49:52.923035 32178 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d5ebd9ba4fb34e66b5c9ddf30309e51c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:49:52.983171 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 31760
I20260809 08:49:52.993881 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 31872
I20260809 08:49:53.003556 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 31977
2026-08-09T08:49:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2869 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:49:53 dist-test-slave-ncp9 krb5kdc[32186](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:49:53 dist-test-slave-ncp9 krb5kdc[32186](info): set up 2 sockets
Aug 09 08:49:53 dist-test-slave-ncp9 krb5kdc[32186](info): commencing operation
krb5kdc: starting...
W20260809 08:49:54.126369 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:49:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:49:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:49:54.236541 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37037
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:37037
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:54.316075 32201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:54.316274 32201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:54.316296 32201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:54.317416 32201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:54.317454 32201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:54.317466 32201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:54.317477 32201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:54.317488 32201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:54.318893 32201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:37037
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:54.319083 32201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:54.319265 32201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:54.323942 32213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:54.323987 32211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:54.324150 32201 server_base.cc:1034] running on GCE node
W20260809 08:49:54.323956 32209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:54.324581 32201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:54.324851 32201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:54.326009 32201 hybrid_clock.cc:648] HybridClock initialized: now 1786265394325999 us; error 26 us; skew 500 ppm
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32186](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265394, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:54.331869 32201 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:54.332278 32201 webserver.cc:460] Webserver started at http://127.10.184.126:42731/ using document root <none> and password file <none>
I20260809 08:49:54.332448 32201 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:54.332484 32201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:54.332573 32201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:54.333624 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "061f8cbb336c46e0b8d720c8acc15d1f"
format_stamp: "Formatted at 2026-08-09 08:49:54 on dist-test-slave-ncp9"
server_key: "9f5eddbebec75990e8975fc6847d2886"
server_key_iv: "ea0407ad03aa1aa8cb30273981b8aaa5"
server_key_version: "encryptionkey@0"
I20260809 08:49:54.333925 32201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "061f8cbb336c46e0b8d720c8acc15d1f"
format_stamp: "Formatted at 2026-08-09 08:49:54 on dist-test-slave-ncp9"
server_key: "9f5eddbebec75990e8975fc6847d2886"
server_key_iv: "ea0407ad03aa1aa8cb30273981b8aaa5"
server_key_version: "encryptionkey@0"
I20260809 08:49:54.335101 32201 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:49:54.335706 32221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:54.335863 32201 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:49:54.335947 32201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "061f8cbb336c46e0b8d720c8acc15d1f"
format_stamp: "Formatted at 2026-08-09 08:49:54 on dist-test-slave-ncp9"
server_key: "9f5eddbebec75990e8975fc6847d2886"
server_key_iv: "ea0407ad03aa1aa8cb30273981b8aaa5"
server_key_version: "encryptionkey@0"
I20260809 08:49:54.336000 32201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:54.369338 32201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:54.371124 32201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:54.371244 32201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:54.374982 32201 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:37037
I20260809 08:49:54.375022 32292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:37037 every 8 connection(s)
I20260809 08:49:54.375347 32201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:54.376188 32294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:54.378444 32294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f: Bootstrap starting.
I20260809 08:49:54.379034 32294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:54.379293 32294 log.cc:824] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:54.379935 32294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f: No bootstrap required, opened a new log
I20260809 08:49:54.380931 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 32201
I20260809 08:49:54.381012 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:49:54.381196 10977 external_mini_cluster.cc:1428] Setting key b574f79494ed73bac2bd75ecae5702ac
I20260809 08:49:54.381673 32294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } }
I20260809 08:49:54.381850 32294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:54.381893 32294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 061f8cbb336c46e0b8d720c8acc15d1f, State: Initialized, Role: FOLLOWER
I20260809 08:49:54.382014 32294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } }
I20260809 08:49:54.382073 32294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:54.382102 32294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:54.382156 32294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:54.383106 32294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } }
I20260809 08:49:54.383224 32294 leader_election.cc:302] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 061f8cbb336c46e0b8d720c8acc15d1f; no voters:
I20260809 08:49:54.383381 32294 leader_election.cc:288] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:49:54.383426 32301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:54.383572 32301 raft_consensus.cc:686] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [term 1 LEADER]: Becoming Leader. State: Replica: 061f8cbb336c46e0b8d720c8acc15d1f, State: Running, Role: LEADER
I20260809 08:49:54.383610 32294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:54.383671 32301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } }
I20260809 08:49:54.384006 32304 sys_catalog.cc:455] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } } }
I20260809 08:49:54.384101 32304 sys_catalog.cc:458] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:54.383991 32301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 061f8cbb336c46e0b8d720c8acc15d1f. Latest consensus state: current_term: 1 leader_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "061f8cbb336c46e0b8d720c8acc15d1f" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37037 } } }
I20260809 08:49:54.384287 32301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:54.384408 32315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:54.385156 32315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:54.387845 32315 catalog_manager.cc:1347] Generated new cluster ID: f00638b52d684adea8ad55aa89d8ed34
I20260809 08:49:54.387899 32315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32186](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265394, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:54.391721 32299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:54.381658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.10.184.126:37037)
0809 08:49:54.381835 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:54.381843 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:54.381867 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:49:54.381983 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:54.381990 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:54.382043 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:54.382163 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:54.382900 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:54.384241 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:54.385361 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:54.386074 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:54.386486 (+ 412us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:54.388725 (+ 2239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:54.388729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:54.388729 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:54.388741 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:54.390243 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:54.390559 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:54.390562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:54.390562 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:54.390657 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:54.390835 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:54.390836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:54.390836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:54.391060 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:54.391218 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:54.391293 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:49:54.391425 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":84,"threads_started":1}
I20260809 08:49:54.392648 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:49:54.409339 32315 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:49:54.410019 32315 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:54.416316 32315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 061f8cbb336c46e0b8d720c8acc15d1f: Generated new TSK 0
I20260809 08:49:54.416468 32315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:49:54.446161 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 32201
2026-08-09T08:49:54Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1444 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32342](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32342](info): set up 2 sockets
Aug 09 08:49:54 dist-test-slave-ncp9 krb5kdc[32342](info): commencing operation
krb5kdc: starting...
W20260809 08:49:55.572681 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:49:55 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:49:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:49:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:49:55.681020 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:55.760219 32361 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:55.760409 32361 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:55.760430 32361 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:55.761526 32361 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:55.761559 32361 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:55.761572 32361 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:55.761583 32361 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:55.761600 32361 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:55.762997 32361 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32361
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:55.763178 32361 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:55.763340 32361 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:55.768006 32371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:55.768085 32374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:55.768033 32369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:55.768091 32361 server_base.cc:1034] running on GCE node
I20260809 08:49:55.768426 32361 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:55.768683 32361 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:55.769816 32361 hybrid_clock.cc:648] HybridClock initialized: now 1786265395769804 us; error 29 us; skew 500 ppm
Aug 09 08:49:55 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:55.772126 32361 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:55.772519 32361 webserver.cc:460] Webserver started at http://127.10.184.126:34901/ using document root <none> and password file <none>
I20260809 08:49:55.772709 32361 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:55.772758 32361 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:55.772964 32361 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:55.773980 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "d719a085549249acb3195ab7e3d48f23"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "604cebfca317f32dd001b01f1dd3a828"
server_key_iv: "a127331964a0e080b10dc9f4972d7a64"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.774320 32361 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "d719a085549249acb3195ab7e3d48f23"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "604cebfca317f32dd001b01f1dd3a828"
server_key_iv: "a127331964a0e080b10dc9f4972d7a64"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.775508 32361 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:49:55.776139 32387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:55.776284 32361 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:49:55.776367 32361 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "d719a085549249acb3195ab7e3d48f23"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "604cebfca317f32dd001b01f1dd3a828"
server_key_iv: "a127331964a0e080b10dc9f4972d7a64"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.776433 32361 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:55.788947 32361 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:55.790584 32361 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:55.790712 32361 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:55.794420 32361 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39155
I20260809 08:49:55.794461 32462 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39155 every 8 connection(s)
I20260809 08:49:55.794762 32361 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:55.795713 32463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:55.798192 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap starting.
I20260809 08:49:55.798768 32463 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:55.799042 32463 log.cc:824] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:55.799690 32463 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: No bootstrap required, opened a new log
I20260809 08:49:55.801328 32463 raft_consensus.cc:348] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:55.801472 32463 raft_consensus.cc:374] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:55.801512 32463 raft_consensus.cc:729] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Initialized, Role: FOLLOWER
I20260809 08:49:55.801661 32463 consensus_queue.cc:260] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:55.801748 32463 raft_consensus.cc:388] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:55.801788 32463 raft_consensus.cc:482] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:55.801839 32463 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:55.802644 32463 raft_consensus.cc:504] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:55.802743 32463 leader_election.cc:302] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d719a085549249acb3195ab7e3d48f23; no voters:
I20260809 08:49:55.802911 32463 leader_election.cc:288] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:49:55.802942 32466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:55.803062 32466 raft_consensus.cc:686] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 LEADER]: Becoming Leader. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Running, Role: LEADER
I20260809 08:49:55.803174 32463 sys_catalog.cc:564] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:55.803189 32466 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:55.803526 32469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719a085549249acb3195ab7e3d48f23. Latest consensus state: current_term: 1 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:55.803625 32469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:55.803524 32467 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:55.803758 32467 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:55.803879 32474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:55.804381 32474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:55.804944 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 32361
I20260809 08:49:55.805074 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:49:55.805235 10977 external_mini_cluster.cc:1428] Setting key 4a66c1d6893dd907fa2b9a3537f98202
I20260809 08:49:55.806075 32474 catalog_manager.cc:1347] Generated new cluster ID: 147610efe0de4bbdb1abcff90e38242e
I20260809 08:49:55.806131 32474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:49:55 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265395, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:55.815380 32474 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:49:55.816008 32474 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:55.816983 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:55.805783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.10.184.126:39155)
0809 08:49:55.805935 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:55.805943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:55.805974 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:49:55.806086 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:55.806093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:55.806147 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:55.806286 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:55.807043 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.808472 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.809525 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.810137 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.810243 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:55.812997 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:55.813002 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:55.813002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:55.813014 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:55.814943 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:55.815334 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:55.815337 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:55.815338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:55.815478 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:55.815711 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:55.815712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:55.815712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:55.816026 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:55.816161 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:55.816582 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 08:49:55.816722 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":71,"threads_started":1}
I20260809 08:49:55.819243 32474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Generated new TSK 0
I20260809 08:49:55.819391 32474 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:49:55.847697 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:55.926930 32496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:55.927124 32496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:55.927145 32496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:55.928249 32496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:55.928289 32496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:55.928314 32496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:55.929766 32496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:55.929975 32496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:55.930182 32496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:55.934975 32503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:55.935000 32496 server_base.cc:1034] running on GCE node
W20260809 08:49:55.934964 32508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:55.934968 32506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:55.935408 32496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:55.935635 32496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:55.936785 32496 hybrid_clock.cc:648] HybridClock initialized: now 1786265395936770 us; error 28 us; skew 500 ppm
Aug 09 08:49:55 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:55.939134 32496 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:55.939532 32496 webserver.cc:460] Webserver started at http://127.10.184.65:34993/ using document root <none> and password file <none>
I20260809 08:49:55.939730 32496 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:55.939779 32496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:55.939905 32496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:55.940919 32496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "e75e7451e4c94eb2ad92d174838580f7"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "d1f1859414841ccfc440a5922b91f0c7"
server_key_iv: "b1616de4e6567b6f0e2179f0257d28fb"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.941267 32496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "e75e7451e4c94eb2ad92d174838580f7"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "d1f1859414841ccfc440a5922b91f0c7"
server_key_iv: "b1616de4e6567b6f0e2179f0257d28fb"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.942454 32496 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:55.943145 32516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:55.943300 32496 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:49:55.943394 32496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "e75e7451e4c94eb2ad92d174838580f7"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "d1f1859414841ccfc440a5922b91f0c7"
server_key_iv: "b1616de4e6567b6f0e2179f0257d28fb"
server_key_version: "encryptionkey@0"
I20260809 08:49:55.943461 32496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:55.971479 32496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:55.973147 32496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:55.973279 32496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:55.973476 32496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:55.973775 32496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:55.973810 32496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:55.973839 32496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:55.973862 32496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:55.979293 32496 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39925
I20260809 08:49:55.979351 32684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39925 every 8 connection(s)
I20260809 08:49:55.979624 32496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:55.981952 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 32496
I20260809 08:49:55.982066 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:49:55.982288 10977 external_mini_cluster.cc:1428] Setting key fbdbafbe3eae36e5ee6a8fb801bbdaed
Aug 09 08:49:55 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265395, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:55.988883 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:55.980301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46869 (local address 127.10.184.126:39155)
0809 08:49:55.980345 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:55.980347 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:55.980548 (+ 201us) server_negotiation.cc:407] Connection header received
0809 08:49:55.981042 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:55.981043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:55.981053 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:55.981101 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:55.981977 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.982690 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.983667 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.983886 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.983942 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:55.986274 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:55.986275 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:55.986276 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:55.986282 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:55.987636 (+ 1354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:55.987999 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:55.988001 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:55.988002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:55.988072 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:55.988283 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:55.988283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:55.988284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:55.988421 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:55.988490 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:55.988745 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 08:49:55.988819 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:49:55.989166 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:55.980287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:46869)
0809 08:49:55.980440 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:55.980474 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:55.980969 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:55.981155 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:55.981159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:55.981699 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:55.981883 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:55.981888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.982773 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.982775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:55.983585 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:55.983588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:55.983954 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:55.983956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:55.984062 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:55.984571 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:55.984582 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:55.986189 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:55.987730 (+ 1541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:55.987737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:55.987739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:55.987942 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:55.988134 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:55.988134 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:55.988135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:55.988230 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:55.988478 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:55.988480 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:55.988616 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 08:49:55.988724 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:49:55.988887 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 08:49:55.989682 32685 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:55.989768 32685 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:55.989928 32685 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:55.990399 32412 ts_manager.cc:194] Registered new tserver with Master: e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65:39925)
I20260809 08:49:55.990967 32412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:46869
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:49:56.007405 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:56.086476 32690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:56.086655 32690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:56.086676 32690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:56.087731 32690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:56.087764 32690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:56.087787 32690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:56.089208 32690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:56.089380 32690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:56.089545 32690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:56.094571 32696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.094584 32695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.094658 32690 server_base.cc:1034] running on GCE node
W20260809 08:49:56.094573 32699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.094930 32690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:56.095175 32690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:56.096315 32690 hybrid_clock.cc:648] HybridClock initialized: now 1786265396096302 us; error 28 us; skew 500 ppm
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:56.098716 32690 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:56.099102 32690 webserver.cc:460] Webserver started at http://127.10.184.66:33825/ using document root <none> and password file <none>
I20260809 08:49:56.099320 32690 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:56.099368 32690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:56.099488 32690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:56.100432 32690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "4155d03b8c81438b930fdac856f6b582"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "4aa37499b751d4aae0dda5da6fa5ff1f"
server_key_iv: "12444afe33f92a01b9e14eceb80a805e"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.100750 32690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "4155d03b8c81438b930fdac856f6b582"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "4aa37499b751d4aae0dda5da6fa5ff1f"
server_key_iv: "12444afe33f92a01b9e14eceb80a805e"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.101976 32690 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:49:56.102695 32708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.102839 32690 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:49:56.102924 32690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4155d03b8c81438b930fdac856f6b582"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "4aa37499b751d4aae0dda5da6fa5ff1f"
server_key_iv: "12444afe33f92a01b9e14eceb80a805e"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.102990 32690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:56.117210 32690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:56.118798 32690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:56.118924 32690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:56.119119 32690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:56.119390 32690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:56.119422 32690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.119449 32690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:56.119467 32690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.124658 32690 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:37367
I20260809 08:49:56.124717 402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:37367 every 8 connection(s)
I20260809 08:49:56.125000 32690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:56.131596 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 32690
I20260809 08:49:56.131712 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:49:56.131906 10977 external_mini_cluster.cc:1428] Setting key 60895eb39d7bfe80caf78ff0458fd535
I20260809 08:49:56.133177 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.125645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:46427 (local address 127.10.184.126:39155)
0809 08:49:56.125721 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.125723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.125888 (+ 165us) server_negotiation.cc:407] Connection header received
0809 08:49:56.126395 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.126396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.126407 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.126454 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.127269 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.127749 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.128627 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.128803 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.128844 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.130672 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.130674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.130674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.130682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.131818 (+ 1136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.132271 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.132273 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.132273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.132336 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.132559 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.132560 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.132560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.132680 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.132732 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.133039 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.133105 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:49:56.133380 407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.125607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:46427)
0809 08:49:56.125764 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.125804 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.126311 (+ 507us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.126503 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.126506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.127030 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:56.127203 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.127208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.127806 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.127807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.128567 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.128569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.128851 (+ 282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.128852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.128987 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.129357 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:56.129364 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:56.130586 (+ 1222us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:56.131897 (+ 1311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.131904 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.131905 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.132188 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.132387 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.132387 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.132388 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.132501 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.132740 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:56.132746 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:56.132914 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.133025 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.133181 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260809 08:49:56.133837 403 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:56.133927 403 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:56.134090 403 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:56.134482 32412 ts_manager.cc:194] Registered new tserver with Master: 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367)
I20260809 08:49:56.134845 32412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:46427
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:49:56.155783 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:56.235538 412 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:56.235729 412 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:56.235760 412 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:56.236871 412 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:56.236940 412 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:56.236974 412 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:56.238410 412 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.412
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:56.238588 412 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:56.238758 412 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:56.243450 419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.243458 418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.243458 423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.243747 412 server_base.cc:1034] running on GCE node
I20260809 08:49:56.243944 412 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:56.244155 412 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:56.245287 412 hybrid_clock.cc:648] HybridClock initialized: now 1786265396245274 us; error 27 us; skew 500 ppm
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:56.247632 412 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:56.248016 412 webserver.cc:460] Webserver started at http://127.10.184.67:45629/ using document root <none> and password file <none>
I20260809 08:49:56.248207 412 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:56.248255 412 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:56.248376 412 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:49:56.249359 412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "e332595cadc449658f173d38f4bff32b"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "419173deb38f20e2407a5c5e82b5309a"
server_key_iv: "54867bb99ad8680a02c2188c4ad6e955"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.249693 412 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "e332595cadc449658f173d38f4bff32b"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "419173deb38f20e2407a5c5e82b5309a"
server_key_iv: "54867bb99ad8680a02c2188c4ad6e955"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.250846 412 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:49:56.251448 433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.251591 412 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:49:56.251683 412 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "e332595cadc449658f173d38f4bff32b"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "419173deb38f20e2407a5c5e82b5309a"
server_key_iv: "54867bb99ad8680a02c2188c4ad6e955"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.251739 412 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:56.284140 412 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:56.286055 412 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:56.286188 412 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:56.286383 412 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:56.286692 412 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:56.286737 412 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.286764 412 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:56.286782 412 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.292315 412 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:40733
I20260809 08:49:56.292352 597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:40733 every 8 connection(s)
I20260809 08:49:56.292635 412 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:56.300163 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 412
I20260809 08:49:56.300279 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:49:56.300480 10977 external_mini_cluster.cc:1428] Setting key 6bbb59f499a50ac86a507674a89f1ab0
I20260809 08:49:56.300518 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.293281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:49907 (local address 127.10.184.126:39155)
0809 08:49:56.293324 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.293326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.293512 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:49:56.293969 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.293970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.293978 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.294029 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.294896 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.295370 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.296261 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.296430 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.296468 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.298364 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.298365 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.298366 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.298373 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.299407 (+ 1034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.299726 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.299727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.299727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.299773 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.299941 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.299942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.299942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.300031 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.300074 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.300393 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.300454 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:49:56.300729 600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.293258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:49907)
0809 08:49:56.293408 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.293448 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.293895 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.294093 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.294096 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.294646 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:56.294829 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.294834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.295429 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.295430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.296200 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.296202 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.296480 (+ 278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.296480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.296568 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.297000 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:56.297009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:56.298274 (+ 1265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:56.299466 (+ 1192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.299472 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.299474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.299668 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.299817 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.299817 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.299818 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.299902 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.300082 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:56.300088 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:56.300257 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.300364 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.300522 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260809 08:49:56.301138 598 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:56.301229 598 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:56.301388 598 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:56.301703 32412 ts_manager.cc:194] Registered new tserver with Master: e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:56.302062 32412 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:49907
I20260809 08:49:56.312449 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:49:56.318889 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.313699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34118 (local address 127.10.184.126:39155)
0809 08:49:56.313738 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.313740 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.313830 (+ 90us) server_negotiation.cc:407] Connection header received
0809 08:49:56.313891 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.313892 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.313902 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.313945 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.314464 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.314913 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.315588 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.315768 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.315818 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.316805 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.316806 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.316806 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.316813 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.317911 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.318198 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.318199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.318199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.318248 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.318404 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.318405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.318405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.318531 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.318618 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.318739 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.318808 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:49:56.320253 32412 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:49:56.321806 32412 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:49:56.327809 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.324017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47796 (local address 127.10.184.66:37367)
0809 08:49:56.324505 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.324508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.324517 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:49:56.324554 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.324556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.324571 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.324792 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:56.324867 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.325923 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.326959 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327435 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327496 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327552 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.327693 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.327732 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":80,"threads_started":1}
I20260809 08:49:56.327800 614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.323803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:37367 (local address 127.0.0.1:47796)
0809 08:49:56.323938 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.323979 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.324045 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.324664 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.324665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.324668 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:56.324716 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.324718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.326005 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.326006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.326878 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.326881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327508 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.327595 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327608 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.327678 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.327731 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":41,"threads_started":1}
I20260809 08:49:56.327960 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.324037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.10.184.65:39925)
0809 08:49:56.324234 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.324237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.324246 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:49:56.324277 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.324278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.324290 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.324361 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:56.324527 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.325421 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.326772 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327488 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327552 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327604 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.327836 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.327878 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":86,"threads_started":1}
I20260809 08:49:56.327993 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.323807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:40733 (local address 127.0.0.1:40418)
0809 08:49:56.324091 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.324100 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.324141 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.324398 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.324399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.324402 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:56.324470 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.324473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.325849 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.325850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.326920 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.326922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327796 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327796 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.327848 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327851 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.327894 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.327924 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":25,"thread_start_us":82,"threads_started":1}
I20260809 08:49:56.328048 615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.323819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39925 (local address 127.0.0.1:34708)
0809 08:49:56.324140 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.324148 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.324196 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.324387 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.324390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.324392 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:56.324452 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.324453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.325516 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.325517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.326678 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.326681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327641 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327643 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.327701 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327706 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.327750 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.327793 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":87,"threads_started":1}
I20260809 08:49:56.328299 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.323948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.10.184.67:40733)
0809 08:49:56.324236 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.324241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.324249 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:56.324280 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.324282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.324295 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.324357 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:56.324620 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.325745 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327001 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.327648 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.327714 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.327764 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.328184 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.328227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":81,"threads_started":1}
I20260809 08:49:56.328773 314 tablet_service.cc:1467] Processing CreateTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (DEFAULT_TABLE table=test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:56.328771 32591 tablet_service.cc:1467] Processing CreateTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (DEFAULT_TABLE table=test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:56.328820 504 tablet_service.cc:1467] Processing CreateTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (DEFAULT_TABLE table=test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:56.329114 504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:56.329130 314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:56.329206 32591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:56.331367 620 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Bootstrap starting.
I20260809 08:49:56.331972 620 tablet_bootstrap.cc:654] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:56.332370 621 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Bootstrap starting.
I20260809 08:49:56.332502 620 log.cc:824] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:56.332984 621 tablet_bootstrap.cc:654] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:56.333289 620 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: No bootstrap required, opened a new log
I20260809 08:49:56.333364 620 ts_tablet_manager.cc:1397] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:49:56.333580 621 log.cc:824] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:56.334295 621 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: No bootstrap required, opened a new log
I20260809 08:49:56.334367 621 ts_tablet_manager.cc:1397] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:49:56.335181 620 raft_consensus.cc:348] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.335383 620 raft_consensus.cc:374] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:56.335439 620 raft_consensus.cc:729] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e332595cadc449658f173d38f4bff32b, State: Initialized, Role: FOLLOWER
I20260809 08:49:56.335551 620 consensus_queue.cc:260] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.335834 620 ts_tablet_manager.cc:1428] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:49:56.335845 598 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
I20260809 08:49:56.336043 624 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Bootstrap starting.
I20260809 08:49:56.336258 621 raft_consensus.cc:348] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.336485 621 raft_consensus.cc:374] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:56.336565 621 raft_consensus.cc:729] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Initialized, Role: FOLLOWER
I20260809 08:49:56.336658 624 tablet_bootstrap.cc:654] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:56.336716 621 consensus_queue.cc:260] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.337100 621 ts_tablet_manager.cc:1428] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:49:56.337107 32685 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
I20260809 08:49:56.337252 624 log.cc:824] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:56.337919 624 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: No bootstrap required, opened a new log
I20260809 08:49:56.337968 624 ts_tablet_manager.cc:1397] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:49:56.339246 624 raft_consensus.cc:348] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.339373 624 raft_consensus.cc:374] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:56.339393 624 raft_consensus.cc:729] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4155d03b8c81438b930fdac856f6b582, State: Initialized, Role: FOLLOWER
I20260809 08:49:56.339493 624 consensus_queue.cc:260] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.339731 624 ts_tablet_manager.cc:1428] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:49:56.339782 403 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
W20260809 08:49:56.375473 404 tablet.cc:2369] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:49:56.456306 629 raft_consensus.cc:482] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:56.456439 629 raft_consensus.cc:504] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.456820 629 leader_election.cc:288] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:56.459421 617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.457089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:46023 (local address 127.10.184.66:37367)
0809 08:49:56.457157 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.457159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.457167 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:56.457193 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.457194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.457205 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.457250 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:56.457382 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.457853 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.458756 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.459120 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.459180 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.459190 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.459326 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.459360 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 08:49:56.459422 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.456988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:37367 (local address 127.10.184.65:46023)
0809 08:49:56.457062 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.457074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.457119 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.457257 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.457258 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.457259 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:56.457326 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.457327 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.457922 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.457923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.458668 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.458670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.459192 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.459193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.459264 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.459267 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.459312 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.459350 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":25}
I20260809 08:49:56.459513 616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.457095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:55619 (local address 127.10.184.67:40733)
0809 08:49:56.457145 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.457147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.457234 (+ 87us) server_negotiation.cc:407] Connection header received
0809 08:49:56.457288 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.457289 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.457298 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.457339 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:56.457483 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.457946 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.458854 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.459217 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.459277 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.459284 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.459418 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.459451 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:49:56.459522 633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.457001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:40733 (local address 127.10.184.65:55619)
0809 08:49:56.457145 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.457154 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.457230 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.457351 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.457352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.457353 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:56.457424 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.457426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.458032 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.458033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.458768 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.458770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.459292 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.459294 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.459354 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.459358 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.459403 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.459440 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":82,"threads_started":1}
I20260809 08:49:56.459856 532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a7d0e0717044d1a4a3fbb69d98d1d0" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332595cadc449658f173d38f4bff32b" is_pre_election: true
I20260809 08:49:56.459856 339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a7d0e0717044d1a4a3fbb69d98d1d0" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4155d03b8c81438b930fdac856f6b582" is_pre_election: true
I20260809 08:49:56.459985 532 raft_consensus.cc:2454] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 0.
I20260809 08:49:56.459992 339 raft_consensus.cc:2454] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 0.
I20260809 08:49:56.460233 32519 leader_election.cc:302] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e332595cadc449658f173d38f4bff32b, e75e7451e4c94eb2ad92d174838580f7; no voters:
I20260809 08:49:56.460368 629 raft_consensus.cc:2781] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:49:56.460438 629 raft_consensus.cc:482] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:49:56.460455 629 raft_consensus.cc:3037] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:56.461167 629 raft_consensus.cc:504] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.461285 629 leader_election.cc:288] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 election: Requested vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:56.461459 339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a7d0e0717044d1a4a3fbb69d98d1d0" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4155d03b8c81438b930fdac856f6b582"
I20260809 08:49:56.461467 532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "83a7d0e0717044d1a4a3fbb69d98d1d0" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332595cadc449658f173d38f4bff32b"
I20260809 08:49:56.461529 532 raft_consensus.cc:3037] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:56.461534 339 raft_consensus.cc:3037] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:56.462231 339 raft_consensus.cc:2454] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 1.
I20260809 08:49:56.462268 532 raft_consensus.cc:2454] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 1.
I20260809 08:49:56.462407 32521 leader_election.cc:302] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4155d03b8c81438b930fdac856f6b582, e75e7451e4c94eb2ad92d174838580f7; no voters:
I20260809 08:49:56.462512 629 raft_consensus.cc:2781] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:56.462652 629 raft_consensus.cc:686] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 LEADER]: Becoming Leader. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Running, Role: LEADER
I20260809 08:49:56.462766 629 consensus_queue.cc:237] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.463574 32412 catalog_manager.cc:5495] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "e75e7451e4c94eb2ad92d174838580f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } health_report { overall_health: HEALTHY } } }
I20260809 08:49:56.472699 618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.470857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34716 (local address 127.10.184.65:39925)
0809 08:49:56.470903 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.470905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.470912 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:49:56.470954 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.470954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.470969 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.471020 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:49:56.471102 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.471511 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.472166 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.472329 (+ 163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.472368 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.472465 (+ 97us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:49:56.472550 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:49:56.472601 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.472615 (+ 14us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.472628 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:49:56.474172 532 raft_consensus.cc:1261] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Refusing update from remote peer e75e7451e4c94eb2ad92d174838580f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:56.474182 339 raft_consensus.cc:1261] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Refusing update from remote peer e75e7451e4c94eb2ad92d174838580f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:56.474459 629 consensus_queue.cc:1035] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:56.474586 634 consensus_queue.cc:1035] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:56.476560 639 mvcc.cc:205] Tried to move back new op lower bound from 7316543063956291584 to 7316543063912140800. Current Snapshot: MvccSnapshot[applied={T|T < 7316543063956291584}]
I20260809 08:49:56.476858 641 mvcc.cc:205] Tried to move back new op lower bound from 7316543063956291584 to 7316543063912140800. Current Snapshot: MvccSnapshot[applied={T|T < 7316543063956291584}]
I20260809 08:49:56.479784 32410 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:56.479878 32410 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:56.480551 32410 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Sending DeleteTablet for 3 replicas of tablet 83a7d0e0717044d1a4a3fbb69d98d1d0
I20260809 08:49:56.480875 504 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:56 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:40418
I20260809 08:49:56.480924 314 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:56 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:47796
I20260809 08:49:56.481033 32591 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:56 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:34708
I20260809 08:49:56.481148 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 32496
I20260809 08:49:56.491591 653 tablet_replica.cc:330] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: stopping tablet replica
I20260809 08:49:56.491714 653 raft_consensus.cc:2229] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:56.491778 655 tablet_replica.cc:330] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: stopping tablet replica
I20260809 08:49:56.491820 653 raft_consensus.cc:2258] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:56.491889 655 raft_consensus.cc:2229] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:56.491991 655 raft_consensus.cc:2258] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:56.492273 653 ts_tablet_manager.cc:1905] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:56.492372 655 ts_tablet_manager.cc:1905] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:56.493857 655 ts_tablet_manager.cc:1918] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:56.493870 653 ts_tablet_manager.cc:1918] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:56.493924 655 log.cc:1196] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/83a7d0e0717044d1a4a3fbb69d98d1d0
I20260809 08:49:56.493943 653 log.cc:1196] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/83a7d0e0717044d1a4a3fbb69d98d1d0
I20260809 08:49:56.494192 655 ts_tablet_manager.cc:1939] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P 4155d03b8c81438b930fdac856f6b582: Deleting consensus metadata
I20260809 08:49:56.494254 653 ts_tablet_manager.cc:1939] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e332595cadc449658f173d38f4bff32b: Deleting consensus metadata
I20260809 08:49:56.494588 32390 catalog_manager.cc:4841] TS 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (table test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]) successfully deleted
I20260809 08:49:56.494661 32388 catalog_manager.cc:4841] TS e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (table test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]) successfully deleted
W20260809 08:49:56.496438 32389 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:39925 (error 108)
I20260809 08:49:56.496658 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 32690
I20260809 08:49:56.501346 613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.496781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39925 (local address 127.0.0.1:34724)
0809 08:49:56.501264 (+ 4483us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.501313 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4459}
W20260809 08:49:56.510687 32389 catalog_manager.cc:4584] TS e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65:39925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39925: connect: Connection refused (error 111)
I20260809 08:49:56.515481 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 412
I20260809 08:49:56.525251 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 32361
I20260809 08:49:56.535351 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--webserver_interface=127.10.184.126
--webserver_port=34901
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:56.614425 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:56.614600 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:56.614620 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:56.615684 661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:56.615715 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:56.615727 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:56.615738 661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:56.615751 661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:56.617115 661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:56.617287 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:56.617447 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:56.626092 668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.626268 673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.626223 669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.626329 661 server_base.cc:1034] running on GCE node
I20260809 08:49:56.626614 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:56.626912 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:56.628043 661 hybrid_clock.cc:648] HybridClock initialized: now 1786265396628034 us; error 27 us; skew 500 ppm
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:56.630399 661 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:56.630851 661 webserver.cc:460] Webserver started at http://127.10.184.126:34901/ using document root <none> and password file <none>
I20260809 08:49:56.631054 661 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:56.631116 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:56.632418 661 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:56.633013 684 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.633167 661 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.633253 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "d719a085549249acb3195ab7e3d48f23"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "604cebfca317f32dd001b01f1dd3a828"
server_key_iv: "a127331964a0e080b10dc9f4972d7a64"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.633631 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:56.645004 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:56.646636 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:56.646772 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:56.650810 661 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39155
I20260809 08:49:56.650839 764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39155 every 8 connection(s)
I20260809 08:49:56.651187 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:56.652770 766 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:49:56.653450 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap starting.
I20260809 08:49:56.659957 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 661
I20260809 08:49:56.660215 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:39925
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=34993
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:56.661593 766 log.cc:824] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:56.663383 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:49:56.663617 766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap complete.
I20260809 08:49:56.665783 766 raft_consensus.cc:348] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:56.666002 766 raft_consensus.cc:729] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Initialized, Role: FOLLOWER
I20260809 08:49:56.666148 766 consensus_queue.cc:260] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:56.666232 766 raft_consensus.cc:388] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:56.666287 766 raft_consensus.cc:482] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:56.666343 766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:49:56.667290 766 raft_consensus.cc:504] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:56.667387 766 leader_election.cc:302] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d719a085549249acb3195ab7e3d48f23; no voters:
I20260809 08:49:56.667541 766 leader_election.cc:288] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:49:56.667589 774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:49:56.667747 774 raft_consensus.cc:686] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 LEADER]: Becoming Leader. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Running, Role: LEADER
I20260809 08:49:56.667804 766 sys_catalog.cc:564] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:56.667872 774 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:56.668162 775 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:56.668256 775 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:56.668393 774 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719a085549249acb3195ab7e3d48f23. Latest consensus state: current_term: 2 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:56.668452 774 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:56.668951 782 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:56.669746 782 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:56.669860 782 catalog_manager.cc:1259] Loaded cluster ID: 147610efe0de4bbdb1abcff90e38242e
I20260809 08:49:56.669898 782 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:49:56.670161 782 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 08:49:56.675187 782 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 08:49:56.677513 782 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:49:56.677702 782 catalog_manager.cc:5879] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Loaded TSK: 0
I20260809 08:49:56.677911 782 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:49:56.744194 768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:56.744374 768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:56.744395 768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:56.745549 768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:56.745591 768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:56.745615 768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:56.747012 768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:39925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=34993
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:56.747215 768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:56.747426 768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:56.753192 809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.753193 811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.753469 814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.753589 768 server_base.cc:1034] running on GCE node
I20260809 08:49:56.753780 768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:56.753989 768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:56.755131 768 hybrid_clock.cc:648] HybridClock initialized: now 1786265396755115 us; error 32 us; skew 500 ppm
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:56.757472 768 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:56.757885 768 webserver.cc:460] Webserver started at http://127.10.184.65:34993/ using document root <none> and password file <none>
I20260809 08:49:56.758102 768 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:56.758159 768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:56.759402 768 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:56.760011 824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.760174 768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:49:56.760280 768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "e75e7451e4c94eb2ad92d174838580f7"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "d1f1859414841ccfc440a5922b91f0c7"
server_key_iv: "b1616de4e6567b6f0e2179f0257d28fb"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.760703 768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:56.777796 768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:56.780001 768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:56.780138 768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:56.780362 768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:56.780786 832 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:49:56.782325 768 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:49:56.782382 768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:49:56.782413 768 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:49:56.783241 768 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:49:56.783282 768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:56.783353 832 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Bootstrap starting.
I20260809 08:49:56.790365 768 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39925
I20260809 08:49:56.790560 1012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39925 every 8 connection(s)
I20260809 08:49:56.791859 768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:56.793391 832 log.cc:824] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:56.795763 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 768
I20260809 08:49:56.795964 832 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:49:56.796046 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:37367
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=33825
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:56.796497 832 tablet_bootstrap.cc:492] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Bootstrap complete.
I20260809 08:49:56.797554 832 ts_tablet_manager.cc:1397] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260809 08:49:56.798799 832 raft_consensus.cc:348] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.799180 832 raft_consensus.cc:729] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Initialized, Role: FOLLOWER
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:56.799374 832 consensus_queue.cc:260] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:56.799754 832 ts_tablet_manager.cc:1428] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:49:56.803254 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.792540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:52869)
0809 08:49:56.792719 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.792768 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.793396 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.794088 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.794092 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.794362 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:56.794548 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.794554 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.795677 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.795678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.796844 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.796848 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.797615 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.797616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.797769 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.798293 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:56.798303 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:56.799836 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:56.801810 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.801817 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.801819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.802038 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.802318 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.802318 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.802318 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.802406 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.802825 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:56.802829 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:56.802931 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.803000 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.803112 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260809 08:49:56.803404 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.792634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:52869 (local address 127.10.184.126:39155)
0809 08:49:56.793682 (+ 1048us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.793691 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.793729 (+ 38us) server_negotiation.cc:407] Connection header received
0809 08:49:56.793860 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.793867 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.793923 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.794047 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.794668 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.795525 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.796965 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.797505 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.798016 (+ 511us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.799954 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.799958 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.799958 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.799970 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.801727 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.802119 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.802122 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.802123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.802251 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.802461 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.802462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.802462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.802758 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.802839 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.803078 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.803215 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":997,"thread_start_us":80,"threads_started":1}
I20260809 08:49:56.804026 1013 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:56.804121 1013 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:56.804283 1013 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:56.805043 714 ts_manager.cc:194] Registered new tserver with Master: e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65:39925)
I20260809 08:49:56.806205 714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:52869
I20260809 08:49:56.807010 1013 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:49:56.813503 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.805630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39925 (local address 127.0.0.1:34732)
0809 08:49:56.805859 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.805884 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.805937 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.806147 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.806152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.806424 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:56.806473 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.806475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.807756 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.807756 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.808302 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.808304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.808931 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.808932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.808986 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.809403 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:56.809407 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:56.810740 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:56.812439 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.812444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.812446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.812643 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.812837 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.812837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.812838 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.812928 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.813183 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:56.813187 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:56.813225 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.813348 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.813439 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":80,"threads_started":1}
I20260809 08:49:56.813560 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.805707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34732 (local address 127.10.184.65:39925)
0809 08:49:56.806026 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.806029 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.806036 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:49:56.806059 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.806060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.806072 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.806124 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.806668 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.807680 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.808381 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.808830 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.808934 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.810820 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.810828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.810828 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.810835 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.812370 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.812719 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.812722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.812722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.812780 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.812967 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.812968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.812968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.813127 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.813205 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.813379 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.813469 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":61,"threads_started":1}
I20260809 08:49:56.813860 905 tablet_service.cc:1514] Processing DeleteTablet for tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:56 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:34732
I20260809 08:49:56.814316 1027 tablet_replica.cc:330] stopping tablet replica
I20260809 08:49:56.814420 1027 raft_consensus.cc:2229] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:56.814481 1027 raft_consensus.cc:2258] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:56.814770 1027 ts_tablet_manager.cc:1905] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:56.816999 1027 ts_tablet_manager.cc:1918] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:56.817072 1027 log.cc:1196] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/83a7d0e0717044d1a4a3fbb69d98d1d0
I20260809 08:49:56.817291 1027 ts_tablet_manager.cc:1939] T 83a7d0e0717044d1a4a3fbb69d98d1d0 P e75e7451e4c94eb2ad92d174838580f7: Deleting consensus metadata
I20260809 08:49:56.817749 687 catalog_manager.cc:4841] TS e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65:39925): tablet 83a7d0e0717044d1a4a3fbb69d98d1d0 (table test-table [id=f9253c2cee8b4f689a264ddd5d9fde01]) successfully deleted
W20260809 08:49:56.881259 1019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:56.881436 1019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:56.881456 1019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:56.882565 1019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:56.882607 1019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:56.882635 1019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:56.884029 1019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:37367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=33825
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:56.884232 1019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:56.884429 1019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:56.889202 1035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.889225 1019 server_base.cc:1034] running on GCE node
W20260809 08:49:56.889173 1034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:56.889323 1040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:56.889485 1019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:56.889729 1019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:56.890858 1019 hybrid_clock.cc:648] HybridClock initialized: now 1786265396890851 us; error 27 us; skew 500 ppm
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:56.893113 1019 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:56.893471 1019 webserver.cc:460] Webserver started at http://127.10.184.66:33825/ using document root <none> and password file <none>
I20260809 08:49:56.893651 1019 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:56.893702 1019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:56.894868 1019 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:49:56.895367 1048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.895506 1019 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.895578 1019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4155d03b8c81438b930fdac856f6b582"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "4aa37499b751d4aae0dda5da6fa5ff1f"
server_key_iv: "12444afe33f92a01b9e14eceb80a805e"
server_key_version: "encryptionkey@0"
I20260809 08:49:56.895877 1019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:56.918316 1019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:56.920083 1019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:56.920225 1019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:56.920439 1019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:56.920723 1019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:56.920758 1019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.920785 1019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:56.920809 1019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:56.926379 1019 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:37367
I20260809 08:49:56.926432 1227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:37367 every 8 connection(s)
I20260809 08:49:56.926746 1019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
Aug 09 08:49:56 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265396, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:56.933140 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 1019
I20260809 08:49:56.933379 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:40733
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=45629
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:56.935585 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.927468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:55757 (local address 127.10.184.126:39155)
0809 08:49:56.927521 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:56.927523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:56.927753 (+ 230us) server_negotiation.cc:407] Connection header received
0809 08:49:56.928250 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:56.928251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:56.928266 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:56.928313 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:56.929269 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.929768 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.930732 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.930940 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.930995 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.932912 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:56.932914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:56.932914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:56.932922 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:56.934220 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.934576 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.934578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.934578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.934633 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:56.934962 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:56.934962 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:56.934963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:56.935097 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:56.935173 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:56.935405 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 08:49:56.935493 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:49:56.935715 1231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:56.927448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:55757)
0809 08:49:56.927621 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:56.927666 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:56.928151 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:56.928391 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:56.928394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:56.928980 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:56.929177 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.929183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.929857 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.929857 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.930639 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:56.930642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:56.931008 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:56.931009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:56.931123 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:56.931511 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:56.931519 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:56.932803 (+ 1284us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:56.934295 (+ 1492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.934302 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.934304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.934510 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.934727 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:56.934728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:56.934729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:56.934890 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:56.935169 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:56.935174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:56.935290 (+ 116us) client_negotiation.cc:769] Sending connection context
0809 08:49:56.935373 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 08:49:56.935498 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260809 08:49:56.936257 1229 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:56.936349 1229 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:56.936514 1229 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:56.936945 714 ts_manager.cc:194] Registered new tserver with Master: 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367)
I20260809 08:49:56.937342 714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:55757
W20260809 08:49:57.013794 1232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:57.013991 1232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:57.014020 1232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:57.015219 1232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:57.015262 1232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:57.015292 1232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:57.016721 1232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:40733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=45629
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:57.016952 1232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:57.017155 1232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:57.021844 1239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.021854 1238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.021878 1241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.021880 1232 server_base.cc:1034] running on GCE node
I20260809 08:49:57.022189 1232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:57.022393 1232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:57.023527 1232 hybrid_clock.cc:648] HybridClock initialized: now 1786265397023519 us; error 27 us; skew 500 ppm
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:57.025815 1232 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:57.026185 1232 webserver.cc:460] Webserver started at http://127.10.184.67:45629/ using document root <none> and password file <none>
I20260809 08:49:57.026371 1232 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:57.026427 1232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:57.027616 1232 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:49:57.028095 1254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.028244 1232 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:49:57.028328 1232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "e332595cadc449658f173d38f4bff32b"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "419173deb38f20e2407a5c5e82b5309a"
server_key_iv: "54867bb99ad8680a02c2188c4ad6e955"
server_key_version: "encryptionkey@0"
I20260809 08:49:57.028602 1232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:57.037396 1232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:57.038971 1232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:57.039067 1232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:57.039216 1232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:57.039431 1232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:57.039458 1232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.039476 1232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:57.039492 1232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.044564 1232 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:40733
I20260809 08:49:57.044624 1425 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:40733 every 8 connection(s)
I20260809 08:49:57.044875 1232 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:49:57.048991 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 1232
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:49:57.053617 1017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.045530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:41401 (local address 127.10.184.126:39155)
0809 08:49:57.045573 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.045575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.045773 (+ 198us) server_negotiation.cc:407] Connection header received
0809 08:49:57.046209 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.046210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.046223 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.046270 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:57.047118 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.047628 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.048555 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.049007 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.049098 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.051053 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:57.051054 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:57.051054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:57.051062 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:57.052353 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:57.052703 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:57.052704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:57.052705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:57.052794 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:57.053041 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:57.053042 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:57.053042 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:57.053146 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:57.053201 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.053442 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.053528 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:49:57.053790 1430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.045517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:41401)
0809 08:49:57.045675 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.045705 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.046119 (+ 414us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.046344 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:57.046348 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:57.046857 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:57.047027 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.047031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.047712 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.047713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.048475 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.048478 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.049112 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.049114 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.049251 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.049696 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:57.049707 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:57.050966 (+ 1259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:57.052423 (+ 1457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:57.052429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:57.052431 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:57.052638 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:57.052847 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:57.052848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:57.052848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:57.052985 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:57.053201 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:57.053208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:57.053330 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 08:49:57.053411 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 08:49:57.053559 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":80,"threads_started":1}
I20260809 08:49:57.054209 1426 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39155
I20260809 08:49:57.054303 1426 heartbeater.cc:464] Registering TS with master...
I20260809 08:49:57.054479 1426 heartbeater.cc:511] Master 127.10.184.126:39155 requested a full tablet report, sending...
I20260809 08:49:57.054845 714 ts_manager.cc:194] Registered new tserver with Master: e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:57.055120 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.049326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34124 (local address 127.10.184.126:39155)
0809 08:49:57.049422 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.049424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.049502 (+ 78us) server_negotiation.cc:407] Connection header received
0809 08:49:57.049565 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.049566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.049578 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.049631 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:57.050145 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.050620 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.051335 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.051597 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.051667 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.052692 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:57.052694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:57.052694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:57.052701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:57.054099 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:57.054426 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:57.054427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:57.054427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:57.054483 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:57.054677 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:57.054678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:57.054678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:57.054801 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:57.054914 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.054993 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.055059 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":49,"threads_started":1}
I20260809 08:49:57.055297 714 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:41401
I20260809 08:49:57.055904 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:49:57.058212 1433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.056205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34132 (local address 127.10.184.126:39155)
0809 08:49:57.056253 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.056254 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.056303 (+ 49us) server_negotiation.cc:407] Connection header received
0809 08:49:57.056357 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.056357 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.056379 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.056431 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:49:57.056498 (+ 67us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.056954 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.057631 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.057807 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.057864 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.057998 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:49:57.058085 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:49:57.058138 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.058155 (+ 17us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.058171 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:49:57.058456 714 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:49:57.060037 714 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:49:57.061911 905 tablet_service.cc:1467] Processing CreateTablet for tablet b86f159633bd4807bc35f8bab69402bd (DEFAULT_TABLE table=test-table [id=2cee381e84094bcf98810751620567c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:57.062114 905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86f159633bd4807bc35f8bab69402bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:57.063313 832 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Bootstrap starting.
I20260809 08:49:57.063648 832 tablet_bootstrap.cc:654] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:57.064153 832 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: No bootstrap required, opened a new log
I20260809 08:49:57.064213 832 ts_tablet_manager.cc:1397] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:57.064539 832 raft_consensus.cc:348] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.064602 832 raft_consensus.cc:374] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:57.064648 832 raft_consensus.cc:729] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Initialized, Role: FOLLOWER
I20260809 08:49:57.064687 832 consensus_queue.cc:260] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.064774 832 ts_tablet_manager.cc:1428] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:57.065805 1024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.061660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:40733 (local address 127.0.0.1:40432)
0809 08:49:57.061753 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.061762 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.061798 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.062145 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:57.062146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:57.062148 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:57.062199 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.062201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.063426 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.063428 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.064615 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.064618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.065535 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.065536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.065626 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.065630 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:57.065673 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:49:57.065717 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":2}
I20260809 08:49:57.065810 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.061661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:37367 (local address 127.0.0.1:47804)
0809 08:49:57.061924 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.061933 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.061990 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.062250 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:57.062251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:57.062253 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:57.062290 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.062292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.063519 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.063520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.064615 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.064618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.065509 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.065510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.065572 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.065594 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 08:49:57.065648 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:49:57.065687 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":15,"thread_start_us":86,"threads_started":1}
I20260809 08:49:57.065882 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.061758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47804 (local address 127.10.184.66:37367)
0809 08:49:57.061904 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.061908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.062044 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:49:57.062083 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.062085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.062098 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.062172 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:57.062363 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.063429 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.064710 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.065437 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.065510 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.065566 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.065767 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.065808 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260809 08:49:57.066228 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.061862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.10.184.67:40733)
0809 08:49:57.061998 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.062002 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.062012 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:49:57.062043 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.062044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.062059 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.062133 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:57.062257 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.063315 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.064723 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.065437 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.066030 (+ 593us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.066100 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.066124 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.066160 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":73,"threads_started":1}
I20260809 08:49:57.066566 1123 tablet_service.cc:1467] Processing CreateTablet for tablet b86f159633bd4807bc35f8bab69402bd (DEFAULT_TABLE table=test-table [id=2cee381e84094bcf98810751620567c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:57.066725 1329 tablet_service.cc:1467] Processing CreateTablet for tablet b86f159633bd4807bc35f8bab69402bd (DEFAULT_TABLE table=test-table [id=2cee381e84094bcf98810751620567c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:49:57.066848 1123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86f159633bd4807bc35f8bab69402bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:57.067006 1329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b86f159633bd4807bc35f8bab69402bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:49:57.069624 1442 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Bootstrap starting.
I20260809 08:49:57.069815 1443 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Bootstrap starting.
I20260809 08:49:57.070231 1442 tablet_bootstrap.cc:654] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:57.070336 1443 tablet_bootstrap.cc:654] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Neither blocks nor log segments found. Creating new log.
I20260809 08:49:57.070767 1442 log.cc:824] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:57.070834 1443 log.cc:824] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:57.071477 1442 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: No bootstrap required, opened a new log
I20260809 08:49:57.071484 1443 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: No bootstrap required, opened a new log
I20260809 08:49:57.071530 1442 ts_tablet_manager.cc:1397] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:49:57.071534 1443 ts_tablet_manager.cc:1397] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:49:57.072774 1443 raft_consensus.cc:348] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.072769 1442 raft_consensus.cc:348] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.072926 1443 raft_consensus.cc:374] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:57.072937 1442 raft_consensus.cc:374] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:49:57.072973 1443 raft_consensus.cc:729] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e332595cadc449658f173d38f4bff32b, State: Initialized, Role: FOLLOWER
I20260809 08:49:57.072979 1442 raft_consensus.cc:729] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4155d03b8c81438b930fdac856f6b582, State: Initialized, Role: FOLLOWER
I20260809 08:49:57.073093 1443 consensus_queue.cc:260] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.073093 1442 consensus_queue.cc:260] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.073340 1443 ts_tablet_manager.cc:1428] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:49:57.073410 1442 ts_tablet_manager.cc:1428] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:49:57.073423 1229 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
I20260809 08:49:57.073345 1426 heartbeater.cc:503] Master 127.10.184.126:39155 was elected leader, sending a full tablet report...
I20260809 08:49:57.121049 1020 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:57.121196 1020 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.121551 1020 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:57.124162 1438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.121786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:47373 (local address 127.10.184.66:37367)
0809 08:49:57.121829 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.121831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.121839 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:57.121865 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.121866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.121875 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.121927 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:57.122071 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.122551 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.123458 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.123835 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.123897 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.123907 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.124065 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.124099 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:49:57.124168 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.121686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:37367 (local address 127.10.184.65:47373)
0809 08:49:57.121740 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.121752 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.121804 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.121936 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:57.121937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:57.121938 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:57.122011 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.122013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.122628 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.122629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.123368 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.123370 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.123910 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.123912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.123982 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.124000 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 08:49:57.124046 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:49:57.124088 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 08:49:57.124264 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.121817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:52641 (local address 127.10.184.67:40733)
0809 08:49:57.121895 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.121897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.121948 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:49:57.122032 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.122033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.122042 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.122082 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:49:57.122259 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.122705 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.123584 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.123948 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.124006 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.124014 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.124171 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.124206 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 08:49:57.124265 1449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.121708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:40733 (local address 127.10.184.65:52641)
0809 08:49:57.121891 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.121901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.121973 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.122096 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:57.122096 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:57.122097 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:49:57.122192 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.122194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.122780 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.122781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.123496 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:57.123498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.124026 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.124027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:57.124101 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.124105 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:49:57.124149 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:49:57.124186 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":124,"threads_started":1}
I20260809 08:49:57.124575 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86f159633bd4807bc35f8bab69402bd" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4155d03b8c81438b930fdac856f6b582" is_pre_election: true
I20260809 08:49:57.124605 1362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86f159633bd4807bc35f8bab69402bd" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332595cadc449658f173d38f4bff32b" is_pre_election: true
I20260809 08:49:57.124752 1362 raft_consensus.cc:2454] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 0.
I20260809 08:49:57.124759 1153 raft_consensus.cc:2454] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 0.
I20260809 08:49:57.125008 825 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e332595cadc449658f173d38f4bff32b, e75e7451e4c94eb2ad92d174838580f7; no voters:
I20260809 08:49:57.125159 1020 raft_consensus.cc:2781] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:49:57.125216 1020 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:49:57.125238 1020 raft_consensus.cc:3037] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:57.125903 1020 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.126035 1020 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 election: Requested vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
I20260809 08:49:57.126196 1362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86f159633bd4807bc35f8bab69402bd" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e332595cadc449658f173d38f4bff32b"
I20260809 08:49:57.126194 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b86f159633bd4807bc35f8bab69402bd" candidate_uuid: "e75e7451e4c94eb2ad92d174838580f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4155d03b8c81438b930fdac856f6b582"
I20260809 08:49:57.126271 1153 raft_consensus.cc:3037] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:57.126271 1362 raft_consensus.cc:3037] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:49:57.127121 1362 raft_consensus.cc:2454] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 1.
I20260809 08:49:57.127121 1153 raft_consensus.cc:2454] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e75e7451e4c94eb2ad92d174838580f7 in term 1.
I20260809 08:49:57.127311 827 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4155d03b8c81438b930fdac856f6b582, e75e7451e4c94eb2ad92d174838580f7; no voters:
I20260809 08:49:57.127436 1020 raft_consensus.cc:2781] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:49:57.127586 1020 raft_consensus.cc:686] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 LEADER]: Becoming Leader. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Running, Role: LEADER
I20260809 08:49:57.127655 1020 consensus_queue.cc:237] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.128379 713 catalog_manager.cc:5495] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65). New cstate: current_term: 1 leader_uuid: "e75e7451e4c94eb2ad92d174838580f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } health_report { overall_health: HEALTHY } } }
I20260809 08:49:57.137889 1025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.135782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34742 (local address 127.10.184.65:39925)
0809 08:49:57.135830 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:57.135832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:57.135840 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:49:57.135884 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:57.135885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:57.135899 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:57.135952 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:49:57.136044 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.136529 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.137267 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:57.137477 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:57.137525 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:57.137654 (+ 129us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:49:57.137743 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:49:57.137788 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:57.137817 (+ 29us) server_negotiation.cc:299] Negotiation successful
0809 08:49:57.137833 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:49:57.139206 1362 raft_consensus.cc:1261] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Refusing update from remote peer e75e7451e4c94eb2ad92d174838580f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:57.139207 1153 raft_consensus.cc:1261] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Refusing update from remote peer e75e7451e4c94eb2ad92d174838580f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:49:57.139511 1450 consensus_queue.cc:1035] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:57.139628 1020 consensus_queue.cc:1035] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:49:57.141928 1452 mvcc.cc:205] Tried to move back new op lower bound from 7316543066679984128 to 7316543066635403264. Current Snapshot: MvccSnapshot[applied={T|T < 7316543066679984128}]
I20260809 08:49:57.142125 1455 mvcc.cc:205] Tried to move back new op lower bound from 7316543066679984128 to 7316543066635403264. Current Snapshot: MvccSnapshot[applied={T|T < 7316543066679984128}]
I20260809 08:49:57.142246 1453 mvcc.cc:205] Tried to move back new op lower bound from 7316543066679984128 to 7316543066635403264. Current Snapshot: MvccSnapshot[applied={T|T < 7316543066679984128}]
I20260809 08:49:57.144196 713 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:57.144292 713 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:49:57.145063 713 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Sending DeleteTablet for 3 replicas of tablet b86f159633bd4807bc35f8bab69402bd
I20260809 08:49:57.145388 1123 tablet_service.cc:1514] Processing DeleteTablet for tablet b86f159633bd4807bc35f8bab69402bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:57 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:47804
I20260809 08:49:57.145402 1329 tablet_service.cc:1514] Processing DeleteTablet for tablet b86f159633bd4807bc35f8bab69402bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:57 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:40432
I20260809 08:49:57.145620 1467 tablet_replica.cc:330] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: stopping tablet replica
I20260809 08:49:57.145638 905 tablet_service.cc:1514] Processing DeleteTablet for tablet b86f159633bd4807bc35f8bab69402bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:49:57 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:34732
I20260809 08:49:57.145715 1027 tablet_replica.cc:330] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: stopping tablet replica
I20260809 08:49:57.145725 1467 raft_consensus.cc:2229] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:57.145764 1027 raft_consensus.cc:2229] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:49:57.145823 1467 raft_consensus.cc:2258] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:57.145845 1027 raft_consensus.cc:2258] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:57.145985 1469 tablet_replica.cc:330] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: stopping tablet replica
I20260809 08:49:57.146076 1469 raft_consensus.cc:2229] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:49:57.146176 1469 raft_consensus.cc:2258] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:49:57.146174 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 768
I20260809 08:49:57.146183 1027 ts_tablet_manager.cc:1905] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:57.149879 1469 ts_tablet_manager.cc:1905] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:57.150758 1467 ts_tablet_manager.cc:1905] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:49:57.151481 1469 ts_tablet_manager.cc:1918] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:57.151584 1469 log.cc:1196] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/b86f159633bd4807bc35f8bab69402bd
I20260809 08:49:57.151916 1469 ts_tablet_manager.cc:1939] T b86f159633bd4807bc35f8bab69402bd P 4155d03b8c81438b930fdac856f6b582: Deleting consensus metadata
I20260809 08:49:57.152295 1467 ts_tablet_manager.cc:1918] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:49:57.152391 688 catalog_manager.cc:4841] TS 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): tablet b86f159633bd4807bc35f8bab69402bd (table test-table [id=2cee381e84094bcf98810751620567c2]) successfully deleted
I20260809 08:49:57.153023 1467 log.cc:1196] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/b86f159633bd4807bc35f8bab69402bd
I20260809 08:49:57.153333 1467 ts_tablet_manager.cc:1939] T b86f159633bd4807bc35f8bab69402bd P e332595cadc449658f173d38f4bff32b: Deleting consensus metadata
I20260809 08:49:57.153749 685 catalog_manager.cc:4841] TS e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): tablet b86f159633bd4807bc35f8bab69402bd (table test-table [id=2cee381e84094bcf98810751620567c2]) successfully deleted
W20260809 08:49:57.158631 687 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:39925 (error 108)
I20260809 08:49:57.158821 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 1019
I20260809 08:49:57.161098 1437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.158965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39925 (local address 127.0.0.1:34754)
0809 08:49:57.161018 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.161067 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39925: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2031}
W20260809 08:49:57.162740 687 catalog_manager.cc:4584] TS e75e7451e4c94eb2ad92d174838580f7 (127.10.184.65:39925): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b86f159633bd4807bc35f8bab69402bd: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39925: connect: Connection refused (error 111)
I20260809 08:49:57.165925 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 1232
I20260809 08:49:57.174546 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 661
I20260809 08:49:57.186100 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--webserver_interface=127.10.184.126
--webserver_port=34901
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39155
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:57.266402 1472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:57.266583 1472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:57.266603 1472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:57.267724 1472 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:49:57.267764 1472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:57.267776 1472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:57.267789 1472 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:49:57.267813 1472 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:49:57.269271 1472 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=34901
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:57.269485 1472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:57.269690 1472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:57.274326 1481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.274304 1478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.274390 1472 server_base.cc:1034] running on GCE node
W20260809 08:49:57.274344 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.274722 1472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:57.274989 1472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:57.276118 1472 hybrid_clock.cc:648] HybridClock initialized: now 1786265397276112 us; error 29 us; skew 500 ppm
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:57.278496 1472 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:49:57.278903 1472 webserver.cc:460] Webserver started at http://127.10.184.126:34901/ using document root <none> and password file <none>
I20260809 08:49:57.279110 1472 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:57.279163 1472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:57.280354 1472 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:57.280952 1489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.281102 1472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.281194 1472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "d719a085549249acb3195ab7e3d48f23"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "604cebfca317f32dd001b01f1dd3a828"
server_key_iv: "a127331964a0e080b10dc9f4972d7a64"
server_key_version: "encryptionkey@0"
I20260809 08:49:57.281497 1472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:57.291316 1472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:57.293289 1472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:57.293412 1472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:57.297091 1472 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39155
I20260809 08:49:57.297137 1567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39155 every 8 connection(s)
I20260809 08:49:57.297469 1472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:49:57.298897 1569 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:49:57.299476 1569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap starting.
I20260809 08:49:57.300391 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 1472
I20260809 08:49:57.300629 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:39925
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=34993
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:49:57.307648 1569 log.cc:824] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:57.309849 1569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:49:57.310096 1569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23: Bootstrap complete.
I20260809 08:49:57.312080 1569 raft_consensus.cc:348] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:57.312335 1569 raft_consensus.cc:729] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Initialized, Role: FOLLOWER
I20260809 08:49:57.312493 1569 consensus_queue.cc:260] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:57.312566 1569 raft_consensus.cc:388] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:49:57.312614 1569 raft_consensus.cc:482] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:49:57.312659 1569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:49:57.313652 1569 raft_consensus.cc:504] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:57.313750 1569 leader_election.cc:302] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d719a085549249acb3195ab7e3d48f23; no voters:
I20260809 08:49:57.313891 1569 leader_election.cc:288] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:49:57.313935 1575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:49:57.314043 1575 raft_consensus.cc:686] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [term 3 LEADER]: Becoming Leader. State: Replica: d719a085549249acb3195ab7e3d48f23, State: Running, Role: LEADER
I20260809 08:49:57.314126 1575 consensus_queue.cc:237] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } }
I20260809 08:49:57.314148 1569 sys_catalog.cc:564] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:49:57.314528 1576 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:57.314531 1577 sys_catalog.cc:455] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d719a085549249acb3195ab7e3d48f23. Latest consensus state: current_term: 3 leader_uuid: "d719a085549249acb3195ab7e3d48f23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d719a085549249acb3195ab7e3d48f23" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39155 } } }
I20260809 08:49:57.314618 1576 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:57.314621 1577 sys_catalog.cc:458] T 00000000000000000000000000000000 P d719a085549249acb3195ab7e3d48f23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:49:57.314999 1583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:49:57.315737 1583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:49:57.315842 1583 catalog_manager.cc:1259] Loaded cluster ID: 147610efe0de4bbdb1abcff90e38242e
I20260809 08:49:57.315874 1583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 08:49:57.316241 1583 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff18a0639ad google::LogMessage::Fail() at ??:0
@ 0x7ff18a06688a google::LogMessage::SendToLog() at ??:0
@ 0x7ff18a0634de google::LogMessage::Flush() at ??:0
@ 0x7ff18a064a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff191f840f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff191f8d8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff18bf310d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff18bf26954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff18cd8a6db start_thread at ??:0
@ 0x7ff188b6161f clone at ??:0
W20260809 08:49:57.396138 1571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:57.396318 1571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:57.396339 1571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:57.397472 1571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:57.397511 1571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:57.397535 1571 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:49:57.398927 1571 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:39925
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=34993
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:57.399118 1571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:57.399322 1571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:57.403990 1632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.404059 1635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.403988 1631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.404214 1571 server_base.cc:1034] running on GCE node
I20260809 08:49:57.404414 1571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:57.404631 1571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:57.405791 1571 hybrid_clock.cc:648] HybridClock initialized: now 1786265397405776 us; error 29 us; skew 500 ppm
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:57.408157 1571 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:49:57.408560 1571 webserver.cc:460] Webserver started at http://127.10.184.65:34993/ using document root <none> and password file <none>
I20260809 08:49:57.408771 1571 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:57.408820 1571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:57.410068 1571 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:49:57.410739 1643 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.410915 1571 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:49:57.411005 1571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "e75e7451e4c94eb2ad92d174838580f7"
format_stamp: "Formatted at 2026-08-09 08:49:55 on dist-test-slave-ncp9"
server_key: "d1f1859414841ccfc440a5922b91f0c7"
server_key_iv: "b1616de4e6567b6f0e2179f0257d28fb"
server_key_version: "encryptionkey@0"
I20260809 08:49:57.411391 1571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:57.436050 1571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:57.438643 1571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:57.438807 1571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:57.439079 1571 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:57.439560 1650 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:49:57.441035 1571 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:49:57.441093 1571 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:49:57.441126 1571 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:49:57.441977 1571 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:49:57.442021 1571 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:49:57.442068 1650 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Bootstrap starting.
I20260809 08:49:57.443842 1650 log.cc:824] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Log is configured to *not* fsync() on all Append() calls
I20260809 08:49:57.447249 1650 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:49:57.447513 1650 tablet_bootstrap.cc:492] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Bootstrap complete.
I20260809 08:49:57.447754 1650 ts_tablet_manager.cc:1397] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 08:49:57.448697 1571 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39925
I20260809 08:49:57.449136 1571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:49:57.449848 1650 raft_consensus.cc:348] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.452173 1650 raft_consensus.cc:729] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75e7451e4c94eb2ad92d174838580f7, State: Initialized, Role: FOLLOWER
I20260809 08:49:57.451647 1805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39925 every 8 connection(s)
I20260809 08:49:57.452308 1650 consensus_queue.cc:260] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:57.452582 1650 ts_tablet_manager.cc:1428] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 08:49:57.456938 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 1571
I20260809 08:49:57.457161 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:37367
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=33825
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:57.538206 1812 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:57.538470 1812 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:57.538501 1812 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:57.539706 1812 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:57.539746 1812 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:57.539770 1812 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:49:57.541234 1812 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:37367
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=33825
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1812
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:57.541455 1812 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:57.541659 1812 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:57.542495 1809 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:49:57.451177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:37241)
0809 08:49:57.451748 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.451788 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:57.451930 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:57.542296 (+ 90366us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":533,"thread_start_us":78,"threads_started":1}
W20260809 08:49:57.542709 1806 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39155 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39155: Client connection negotiation failed: client connection to 127.10.184.126:39155: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:49:57.543051 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.542926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:56211)
0809 08:49:57.542979 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.543028 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:49:57.543418 1809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.543320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:42245)
0809 08:49:57.543361 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.543398 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 08:49:57.543540 1806 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:49:57.546722 1818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.546733 1824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.546723 1820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.546806 1812 server_base.cc:1034] running on GCE node
I20260809 08:49:57.547096 1812 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:57.547317 1812 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:57.548456 1812 hybrid_clock.cc:648] HybridClock initialized: now 1786265397548441 us; error 30 us; skew 500 ppm
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:57.550781 1812 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:49:57.551178 1812 webserver.cc:460] Webserver started at http://127.10.184.66:33825/ using document root <none> and password file <none>
I20260809 08:49:57.551384 1812 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:57.551433 1812 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:57.552731 1812 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:57.553323 1835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.553471 1812 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.553555 1812 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "4155d03b8c81438b930fdac856f6b582"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "4aa37499b751d4aae0dda5da6fa5ff1f"
server_key_iv: "12444afe33f92a01b9e14eceb80a805e"
server_key_version: "encryptionkey@0"
I20260809 08:49:57.553872 1812 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:57.573225 1812 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:57.574903 1812 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:57.575037 1812 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:57.575249 1812 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:57.575546 1812 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:57.575587 1812 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.575615 1812 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:57.575634 1812 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.581264 1812 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:37367
I20260809 08:49:57.581322 1995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:37367 every 8 connection(s)
I20260809 08:49:57.581632 1812 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:49:57.582589 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.582219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:58495)
0809 08:49:57.582361 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.582438 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260809 08:49:57.582610 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 1812
I20260809 08:49:57.582880 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:40733
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:39155
--webserver_port=45629
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:49:57.583026 1997 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39155 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39155: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
I20260809 08:49:57.583289 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.583198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:34035)
0809 08:49:57.583239 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.583264 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:49:57.583578 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.583515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:47155)
0809 08:49:57.583546 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.583566 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 08:49:57.583696 1997 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:49:57.663589 2002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:49:57.663792 2002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:49:57.663823 2002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:49:57.664963 2002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:49:57.665004 2002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:49:57.665036 2002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:49:57.666452 2002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:40733
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=45629
--tserver_master_addrs=127.10.184.126:39155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:49:57.666683 2002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:49:57.666889 2002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:49:57.671774 2007 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.671841 2011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:49:57.671787 2014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:49:57.671875 2002 server_base.cc:1034] running on GCE node
I20260809 08:49:57.672171 2002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:49:57.672405 2002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:49:57.673538 2002 hybrid_clock.cc:648] HybridClock initialized: now 1786265397673527 us; error 30 us; skew 500 ppm
Aug 09 08:49:57 dist-test-slave-ncp9 krb5kdc[32342](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:49:57.675853 2002 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:49:57.676222 2002 webserver.cc:460] Webserver started at http://127.10.184.67:45629/ using document root <none> and password file <none>
I20260809 08:49:57.676420 2002 fs_manager.cc:359] Metadata directory not provided
I20260809 08:49:57.676463 2002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:49:57.677696 2002 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:49:57.678213 2027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.678354 2002 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.678426 2002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "e332595cadc449658f173d38f4bff32b"
format_stamp: "Formatted at 2026-08-09 08:49:56 on dist-test-slave-ncp9"
server_key: "419173deb38f20e2407a5c5e82b5309a"
server_key_iv: "54867bb99ad8680a02c2188c4ad6e955"
server_key_version: "encryptionkey@0"
I20260809 08:49:57.678726 2002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:49:57.696619 2002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:49:57.698240 2002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:49:57.698365 2002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:49:57.698571 2002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:49:57.698846 2002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:49:57.698879 2002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.698906 2002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:49:57.698923 2002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:49:57.704304 2002 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:40733
I20260809 08:49:57.704363 2208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:40733 every 8 connection(s)
I20260809 08:49:57.704666 2002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:49:57.705606 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.705271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:43563)
0809 08:49:57.705407 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.705478 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
W20260809 08:49:57.705960 2210 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39155 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39155: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
I20260809 08:49:57.706205 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.706121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:43329)
0809 08:49:57.706163 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.706188 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:49:57.706413 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:57.706354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:34339)
0809 08:49:57.706382 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:57.706400 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
W20260809 08:49:57.706493 2210 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:49:57.707832 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 2002
I20260809 08:49:58.544093 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.543845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:52221)
0809 08:49:58.543982 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:58.544030 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260809 08:49:58.584124 2219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.583942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:45205)
0809 08:49:58.584048 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:58.584077 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":55,"threads_started":1}
I20260809 08:49:58.706954 2220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.706778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:38389)
0809 08:49:58.706881 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:58.706908 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":57,"threads_started":1}
I20260809 08:49:58.795341 2222 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:49:58.795459 2222 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:49:58.795759 2222 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
Aug 09 08:49:58 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.67@KRBTEST.COM
Aug 09 08:49:58 dist-test-slave-ncp9 krb5kdc[32342](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265397, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.66@KRBTEST.COM
I20260809 08:49:58.805480 2225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.796041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:53933 (local address 127.10.184.66:37367)
0809 08:49:58.796209 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:58.796220 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:58.796251 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:49:58.796683 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:58.796685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:58.796723 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:58.796844 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:58.797789 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.798600 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.799537 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.799930 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.800022 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:58.802575 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:58.802578 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:58.802578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:58.802588 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:58.804173 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804455 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:58.804458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:58.804458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:58.804545 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804713 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:58.804714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:58.804714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:58.804975 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:58.805047 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:58.805250 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 08:49:58.805365 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":58,"threads_started":1}
I20260809 08:49:58.805480 2224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.796037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:55521 (local address 127.10.184.67:40733)
0809 08:49:58.796184 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 08:49:58.796197 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:49:58.796229 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:49:58.796679 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:49:58.796681 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:49:58.796723 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:49:58.796837 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:49:58.797641 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.798502 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.799507 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.799910 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.800008 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:58.802226 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:49:58.802230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:49:58.802230 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:49:58.802239 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:49:58.803966 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804342 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:58.804345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:58.804345 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:58.804428 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804653 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:49:58.804654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:49:58.804655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:49:58.804871 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:49:58.804987 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 08:49:58.805262 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 08:49:58.805373 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":44,"threads_started":1}
I20260809 08:49:58.805505 2217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.795924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:37367 (local address 127.10.184.65:53933)
0809 08:49:58.795977 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:58.795988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:58.796042 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:58.796926 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:58.796929 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:58.797297 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:58.797683 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:58.797686 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.798699 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.798700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:58.799426 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:58.799428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.800005 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.800006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:58.800113 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:58.800617 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:58.800625 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:58.802482 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:58.804240 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804241 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:58.804241 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:58.804387 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:58.804605 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:58.804607 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:58.804661 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:58.805050 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:58.805051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:58.805110 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 08:49:58.805196 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 08:49:58.805373 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":9}
I20260809 08:49:58.805574 2223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:58.795929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:40733 (local address 127.10.184.65:55521)
0809 08:49:58.796095 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:58.796105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:49:58.796160 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:49:58.796973 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:49:58.796974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:49:58.797289 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:49:58.797525 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:58.797531 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.798599 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.798599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:58.799424 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:49:58.799427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:49:58.799993 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:49:58.799994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:49:58.800110 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:49:58.800666 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:49:58.800670 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:49:58.802108 (+ 1438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:49:58.804044 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:58.804052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:58.804260 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:58.804489 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:49:58.804489 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:49:58.804490 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:49:58.804590 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:49:58.804978 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:49:58.804985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:49:58.805104 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 08:49:58.805196 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:49:58.805442 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":188,"thread_start_us":56,"threads_started":1}
W20260809 08:49:58.806517 1644 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
W20260809 08:49:58.806627 1646 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
I20260809 08:49:58.806679 1646 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75e7451e4c94eb2ad92d174838580f7; no voters: 4155d03b8c81438b930fdac856f6b582, e332595cadc449658f173d38f4bff32b
I20260809 08:49:58.806777 2222 raft_consensus.cc:2726] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:49:59.544683 2227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:59.544464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:58437)
0809 08:49:59.544595 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:59.544637 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260809 08:49:59.584673 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:59.584465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:37609)
0809 08:49:59.584594 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:59.584628 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260809 08:49:59.707556 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:49:59.707346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:45815)
0809 08:49:59.707479 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:49:59.707520 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260809 08:50:00.545322 2231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:00.545110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:55351)
0809 08:50:00.545240 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:00.545276 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260809 08:50:00.585319 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:00.585133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:46651)
0809 08:50:00.585248 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:00.585283 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260809 08:50:00.708174 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:00.707998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:49715)
0809 08:50:00.708105 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:00.708137 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":62,"threads_started":1}
I20260809 08:50:01.039534 2234 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:01.039624 2234 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:50:01.039813 2234 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
W20260809 08:50:01.040252 1646 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
W20260809 08:50:01.040336 1644 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
I20260809 08:50:01.040413 1644 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75e7451e4c94eb2ad92d174838580f7; no voters: 4155d03b8c81438b930fdac856f6b582, e332595cadc449658f173d38f4bff32b
I20260809 08:50:01.040484 2234 raft_consensus.cc:2726] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:01.546015 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:01.545788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:44675)
0809 08:50:01.545923 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:01.545970 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
I20260809 08:50:01.585879 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:01.585698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:46627)
0809 08:50:01.585813 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:01.585845 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":57,"threads_started":1}
I20260809 08:50:01.708705 2238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:01.708522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:51415)
0809 08:50:01.708637 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:01.708669 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260809 08:50:02.546732 2239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:02.546515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:49643)
0809 08:50:02.546643 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:02.546688 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":66,"threads_started":1}
I20260809 08:50:02.586445 2243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:02.586267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:43173)
0809 08:50:02.586377 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:02.586410 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260809 08:50:02.709281 2244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:02.709093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:54467)
0809 08:50:02.709208 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:02.709243 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":61,"threads_started":1}
I20260809 08:50:03.043038 2245 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:03.043130 2245 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:50:03.043279 2245 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
W20260809 08:50:03.043675 1644 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
W20260809 08:50:03.043772 1646 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
I20260809 08:50:03.043834 1646 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75e7451e4c94eb2ad92d174838580f7; no voters: 4155d03b8c81438b930fdac856f6b582, e332595cadc449658f173d38f4bff32b
I20260809 08:50:03.043922 2245 raft_consensus.cc:2726] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:03.547374 2247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:03.547178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:59855)
0809 08:50:03.547293 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:03.547328 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260809 08:50:03.586973 2249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:03.586797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:60751)
0809 08:50:03.586903 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:03.586937 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":58,"threads_started":1}
I20260809 08:50:03.709822 2250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:03.709644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:55285)
0809 08:50:03.709752 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:03.709784 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":61,"threads_started":1}
I20260809 08:50:04.547993 2252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:04.547779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:48013)
0809 08:50:04.547915 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:04.547950 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260809 08:50:04.587496 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:04.587309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:40887)
0809 08:50:04.587430 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:04.587459 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260809 08:50:04.710388 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:04.710193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:50017)
0809 08:50:04.710317 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:04.710351 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260809 08:50:05.243845 2255 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:05.243940 2255 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:50:05.244119 2255 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
W20260809 08:50:05.244519 1644 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
W20260809 08:50:05.244623 1646 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
I20260809 08:50:05.244700 1646 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75e7451e4c94eb2ad92d174838580f7; no voters: 4155d03b8c81438b930fdac856f6b582, e332595cadc449658f173d38f4bff32b
I20260809 08:50:05.244783 2255 raft_consensus.cc:2726] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:05.548686 2256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:05.548432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:48525)
0809 08:50:05.548584 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:05.548617 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":79,"threads_started":1}
I20260809 08:50:05.588075 2257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:05.587876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:46101)
0809 08:50:05.587985 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:05.588019 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260809 08:50:05.710963 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:05.710758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:33073)
0809 08:50:05.710893 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:05.710922 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 08:50:06.549299 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:06.549075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:33065)
0809 08:50:06.549218 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:06.549255 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260809 08:50:06.588660 2260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:06.588453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:50081)
0809 08:50:06.588570 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:06.588611 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260809 08:50:06.711520 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:06.711321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:55791)
0809 08:50:06.711446 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:06.711484 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260809 08:50:06.840242 2264 raft_consensus.cc:482] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:06.840334 2264 raft_consensus.cc:504] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4155d03b8c81438b930fdac856f6b582" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 37367 } } peers { permanent_uuid: "e332595cadc449658f173d38f4bff32b" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 40733 } } peers { permanent_uuid: "e75e7451e4c94eb2ad92d174838580f7" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39925 } }
I20260809 08:50:06.840502 2264 leader_election.cc:288] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367), e332595cadc449658f173d38f4bff32b (127.10.184.67:40733)
W20260809 08:50:06.840852 1644 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e332595cadc449658f173d38f4bff32b (127.10.184.67:40733): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
W20260809 08:50:06.840960 1646 leader_election.cc:341] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4155d03b8c81438b930fdac856f6b582 (127.10.184.66:37367): Not found: Tablet not found: b86f159633bd4807bc35f8bab69402bd
I20260809 08:50:06.841004 1646 leader_election.cc:302] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e75e7451e4c94eb2ad92d174838580f7; no voters: 4155d03b8c81438b930fdac856f6b582, e332595cadc449658f173d38f4bff32b
I20260809 08:50:06.841077 2264 raft_consensus.cc:2726] T b86f159633bd4807bc35f8bab69402bd P e75e7451e4c94eb2ad92d174838580f7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:07.549938 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:07.549734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:52479)
0809 08:50:07.549860 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:07.549891 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":65,"threads_started":1}
I20260809 08:50:07.589264 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:07.589069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:51499)
0809 08:50:07.589196 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:07.589226 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":58,"threads_started":1}
I20260809 08:50:07.712090 2269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:07.711885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:47621)
0809 08:50:07.712019 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:07.712051 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 08:50:08.550575 2270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:08.550363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:38869)
0809 08:50:08.550498 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:08.550529 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260809 08:50:08.589829 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:08.589624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:44445)
0809 08:50:08.589760 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:08.589791 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260809 08:50:08.712708 2273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:08.712469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:46237)
0809 08:50:08.712624 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:08.712664 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":77,"threads_started":1}
I20260809 08:50:09.551209 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:09.550986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:58405)
0809 08:50:09.551131 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:09.551164 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":77,"threads_started":1}
I20260809 08:50:09.591008 2275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:09.590237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:34385)
0809 08:50:09.590931 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:09.590965 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":639,"threads_started":1}
I20260809 08:50:09.713277 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:09.713073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:44181)
0809 08:50:09.713201 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:09.713236 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":60,"threads_started":1}
I20260809 08:50:10.551836 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:10.551606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:59467)
0809 08:50:10.551753 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:10.551791 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":66,"threads_started":1}
I20260809 08:50:10.591578 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:10.591397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:60903)
0809 08:50:10.591509 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:10.591540 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260809 08:50:10.713951 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:10.713734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:55955)
0809 08:50:10.713876 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:10.713910 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":63,"threads_started":1}
I20260809 08:50:11.552465 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:11.552241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:44249)
0809 08:50:11.552382 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:11.552419 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":74,"threads_started":1}
I20260809 08:50:11.592132 2285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:11.591946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:52287)
0809 08:50:11.592063 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:11.592094 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260809 08:50:11.714579 2287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:11.714340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:37165)
0809 08:50:11.714501 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:11.714537 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260809 08:50:12.553249 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:12.553033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.65:36869)
0809 08:50:12.553159 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:12.553204 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260809 08:50:12.592742 2289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:12.592559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.66:45743)
0809 08:50:12.592675 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:12.592703 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":61,"threads_started":1}
I20260809 08:50:12.712178 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 1571
I20260809 08:50:12.717406 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:12.717026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39155 (local address 127.10.184.67:39865)
0809 08:50:12.717324 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:12.717359 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39155: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":52,"threads_started":1}
I20260809 08:50:12.722625 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 1812
I20260809 08:50:12.731516 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 2002
2026-08-09T08:50:12Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18284 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:50:12 dist-test-slave-ncp9 krb5kdc[2293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:50:12 dist-test-slave-ncp9 krb5kdc[2293](info): set up 2 sockets
Aug 09 08:50:12 dist-test-slave-ncp9 krb5kdc[2293](info): commencing operation
krb5kdc: starting...
W20260809 08:50:13.865037 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:50:13 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:50:13.980087 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:14.071786 2309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:14.071980 2309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:14.072010 2309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:14.073205 2309 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:50:14.073246 2309 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:50:14.073262 2309 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:50:14.073276 2309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:50:14.073292 2309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:14.073318 2309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:14.073341 2309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:50:14.073365 2309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:50:14.074872 2309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:14.075111 2309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:14.075315 2309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:14.080085 2319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:14.080096 2320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.080133 2309 server_base.cc:1034] running on GCE node
W20260809 08:50:14.080240 2325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.080581 2309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:14.080873 2309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:14.082072 2309 hybrid_clock.cc:648] HybridClock initialized: now 1786265414082053 us; error 30 us; skew 500 ppm
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:14.084437 2309 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:50:14.084805 2309 webserver.cc:460] Webserver started at http://127.10.184.126:42127/ using document root <none> and password file <none>
I20260809 08:50:14.085027 2309 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:14.085073 2309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:14.085162 2309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:50:14.086110 2309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "9ea9b01e867448cf8647a344dd982099"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "c7ebb6cd8c69aa3bdcf6482aff67b61d"
server_key_iv: "4e757ccf1f008e63b2c147e0236fd35b"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.086463 2309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "9ea9b01e867448cf8647a344dd982099"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "c7ebb6cd8c69aa3bdcf6482aff67b61d"
server_key_iv: "4e757ccf1f008e63b2c147e0236fd35b"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.087625 2309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:50:14.088259 2335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.088411 2309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:14.088495 2309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9ea9b01e867448cf8647a344dd982099"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "c7ebb6cd8c69aa3bdcf6482aff67b61d"
server_key_iv: "4e757ccf1f008e63b2c147e0236fd35b"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.088560 2309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:14.106910 2309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:14.108922 2309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:14.109067 2309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:14.112645 2309 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39307
I20260809 08:50:14.112682 2427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39307 every 8 connection(s)
I20260809 08:50:14.113041 2309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:50:14.113843 2428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:14.114216 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 2309
I20260809 08:50:14.114334 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:50:14.114524 10977 external_mini_cluster.cc:1428] Setting key edc19ce7a6438011f6dc6200d54d9c37
I20260809 08:50:14.116338 2428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap starting.
I20260809 08:50:14.116910 2428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:14.117199 2428 log.cc:824] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:14.117792 2428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: No bootstrap required, opened a new log
I20260809 08:50:14.119489 2428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:14.119618 2428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:14.119654 2428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Initialized, Role: FOLLOWER
I20260809 08:50:14.119774 2428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:14.119849 2428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:50:14.119881 2428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:50:14.119928 2428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:14.120414 2428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:14.120508 2428 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea9b01e867448cf8647a344dd982099; no voters:
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:14.120663 2428 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:50:14.120710 2434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:50:14.120821 2434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 LEADER]: Becoming Leader. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Running, Role: LEADER
I20260809 08:50:14.120955 2428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:50:14.120988 2434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:14.121393 2434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea9b01e867448cf8647a344dd982099. Latest consensus state: current_term: 1 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:14.121543 2434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:14.121613 2438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:14.121737 2438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:14.122048 2448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:50:14.122530 2448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:50:14.124231 2448 catalog_manager.cc:1347] Generated new cluster ID: f10d41a1a30e40fda7a3988a3d3c81f4
I20260809 08:50:14.124281 2448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:50:14.125414 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.115063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43390 (local address 127.10.184.126:39307)
0809 08:50:14.115236 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.115244 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.115267 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:50:14.115655 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.115661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.115706 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.115831 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:14.116435 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.117497 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.118356 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.118780 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.118886 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.121319 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:14.121323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:14.121324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:14.121334 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:14.123555 (+ 2221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.123920 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.123923 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.123924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.124010 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.124214 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.124216 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.124216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.124499 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:14.125015 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.125080 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.125211 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":75,"threads_started":1}
I20260809 08:50:14.138010 2448 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:50:14.138610 2448 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:50:14.145591 2448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Generated new TSK 0
I20260809 08:50:14.145746 2448 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:50:14.175621 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:14.279260 2466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:14.279444 2466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:14.279465 2466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:14.280516 2466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:14.280547 2466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:14.280572 2466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:50:14.281991 2466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:14.282163 2466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:14.282354 2466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:14.287125 2474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.287143 2466 server_base.cc:1034] running on GCE node
W20260809 08:50:14.287170 2475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:14.287104 2478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.287544 2466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:14.287756 2466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:14.288878 2466 hybrid_clock.cc:648] HybridClock initialized: now 1786265414288865 us; error 31 us; skew 500 ppm
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:14.291260 2466 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:50:14.291651 2466 webserver.cc:460] Webserver started at http://127.10.184.65:35211/ using document root <none> and password file <none>
I20260809 08:50:14.291838 2466 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:14.291883 2466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:14.292006 2466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:50:14.293040 2466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "cf79186ef85a474abb1556f20cf23d62"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "fe165eb42bfbe0c945634f60cfddc4e7"
server_key_iv: "ee90a1bbd35fc113f5c2496b3863af09"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.293366 2466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "cf79186ef85a474abb1556f20cf23d62"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "fe165eb42bfbe0c945634f60cfddc4e7"
server_key_iv: "ee90a1bbd35fc113f5c2496b3863af09"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.294529 2466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:50:14.295136 2494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.295292 2466 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:50:14.295375 2466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "cf79186ef85a474abb1556f20cf23d62"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "fe165eb42bfbe0c945634f60cfddc4e7"
server_key_iv: "ee90a1bbd35fc113f5c2496b3863af09"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.295435 2466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:14.304373 2466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:14.305958 2466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:14.306084 2466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:14.306314 2466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:14.306586 2466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:14.306620 2466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.306645 2466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:14.306663 2466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.311920 2466 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:40763
I20260809 08:50:14.311987 2666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:40763 every 8 connection(s)
I20260809 08:50:14.312242 2466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:14.320322 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.312862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:45721 (local address 127.10.184.126:39307)
0809 08:50:14.312938 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.312941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.313102 (+ 161us) server_negotiation.cc:407] Connection header received
0809 08:50:14.313553 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.313554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.313567 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.313617 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:14.314486 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.315004 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.315954 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.316171 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.316224 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.318117 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:14.318119 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:14.318119 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:14.318126 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:14.319144 (+ 1018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.319602 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.319603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.319603 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.319652 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.319850 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.319851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.319851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.319944 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:14.319987 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.320198 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.320260 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:50:14.320544 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.312812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:45721)
0809 08:50:14.312991 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.313032 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.313457 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.313691 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.313695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.314210 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:14.314384 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.314388 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.315092 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.315093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.315868 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.315870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.316244 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.316245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.316356 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.316745 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:14.316753 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:14.318027 (+ 1274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:14.319263 (+ 1236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.319271 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.319273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.319529 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.319706 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.319707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.319707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.319802 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.319989 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:14.319993 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:14.320097 (+ 104us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.320177 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.320295 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":102,"threads_started":1}
I20260809 08:50:14.320518 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 2466
I20260809 08:50:14.320636 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:50:14.320852 10977 external_mini_cluster.cc:1428] Setting key d43c749e01d1cae36f49654ae5f7eecd
I20260809 08:50:14.321103 2667 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:14.321203 2667 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:14.321388 2667 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
I20260809 08:50:14.321931 2367 ts_manager.cc:194] Registered new tserver with Master: cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:14.322618 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:45721
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:50:14.345396 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:14.425225 2675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:14.425411 2675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:14.425442 2675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:14.426546 2675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:14.426586 2675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:14.426620 2675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:50:14.428007 2675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:14.428227 2675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:14.428424 2675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:14.433132 2685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.433229 2675 server_base.cc:1034] running on GCE node
W20260809 08:50:14.433138 2682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:14.433159 2681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.433514 2675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:14.433756 2675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:14.434906 2675 hybrid_clock.cc:648] HybridClock initialized: now 1786265414434892 us; error 30 us; skew 500 ppm
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:14.437252 2675 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:50:14.437600 2675 webserver.cc:460] Webserver started at http://127.10.184.66:45329/ using document root <none> and password file <none>
I20260809 08:50:14.437767 2675 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:14.437803 2675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:14.437887 2675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:50:14.438788 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "9e3c64bbe89f698314820fb33fbd1038"
server_key_iv: "60218ca3da604eb6ca6ace22c2693957"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.439090 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "9e3c64bbe89f698314820fb33fbd1038"
server_key_iv: "60218ca3da604eb6ca6ace22c2693957"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.440202 2675 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:50:14.440788 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.440963 2675 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.441046 2675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "9e3c64bbe89f698314820fb33fbd1038"
server_key_iv: "60218ca3da604eb6ca6ace22c2693957"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.441099 2675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:14.460004 2675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:14.461648 2675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:14.461755 2675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:14.461923 2675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:14.462212 2675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:14.462247 2675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.462266 2675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:14.462289 2675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.467523 2675 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43039
I20260809 08:50:14.467583 2879 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43039 every 8 connection(s)
I20260809 08:50:14.467851 2675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:50:14.469460 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 2675
I20260809 08:50:14.469552 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:50:14.469722 10977 external_mini_cluster.cc:1428] Setting key b4164e91c2b543a93ea8259915973a12
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:14.477102 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.468482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:33205 (local address 127.10.184.126:39307)
0809 08:50:14.468560 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.468562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.468704 (+ 142us) server_negotiation.cc:407] Connection header received
0809 08:50:14.469188 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.469190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.469200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.469243 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:14.470242 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.470792 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.472100 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.472376 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.472437 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.474511 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:14.474513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:14.474513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:14.474520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:14.475738 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.476190 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.476191 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.476191 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.476237 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.476494 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.476494 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.476495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.476585 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:14.476626 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.476977 (+ 351us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.477040 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 08:50:14.477331 2884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.468434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:33205)
0809 08:50:14.468589 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.468623 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.469105 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.469316 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.469319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.469916 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:14.470151 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.470157 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.470884 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.470885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.472006 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.472010 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.472447 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.472447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.472544 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.473009 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:14.473018 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:14.474421 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:14.475826 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.475834 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.475836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.476118 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.476296 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.476297 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.476298 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.476434 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.476642 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:14.476656 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:14.476809 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.476959 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.477120 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":76,"threads_started":1}
I20260809 08:50:14.477815 2881 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:14.477914 2881 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:14.478096 2881 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
I20260809 08:50:14.478446 2367 ts_manager.cc:194] Registered new tserver with Master: 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
I20260809 08:50:14.478820 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:33205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:50:14.493047 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:14.574321 2889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:14.574519 2889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:14.574553 2889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:14.576090 2889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:14.576148 2889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:14.576184 2889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:50:14.577740 2889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:14.577921 2889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:14.578092 2889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:14.582808 2896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:14.582831 2897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:14.582820 2901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:14.582989 2889 server_base.cc:1034] running on GCE node
I20260809 08:50:14.583281 2889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:14.583495 2889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:14.584628 2889 hybrid_clock.cc:648] HybridClock initialized: now 1786265414584615 us; error 29 us; skew 500 ppm
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:14.587169 2889 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:50:14.587600 2889 webserver.cc:460] Webserver started at http://127.10.184.67:40925/ using document root <none> and password file <none>
I20260809 08:50:14.587808 2889 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:14.587855 2889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:14.587972 2889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:50:14.589020 2889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "cc0658909f704494abff0400492e39c9"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "bf1f543914fbfedc5358c77fba97eb60"
server_key_iv: "0af70fbdad39d947e8c0cc854538fb89"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.589380 2889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "cc0658909f704494abff0400492e39c9"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "bf1f543914fbfedc5358c77fba97eb60"
server_key_iv: "0af70fbdad39d947e8c0cc854538fb89"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.590698 2889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:14.591326 2911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.591499 2889 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.591583 2889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "cc0658909f704494abff0400492e39c9"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "bf1f543914fbfedc5358c77fba97eb60"
server_key_iv: "0af70fbdad39d947e8c0cc854538fb89"
server_key_version: "encryptionkey@0"
I20260809 08:50:14.591665 2889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:14.604121 2889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:14.605918 2889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:14.606045 2889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:14.606233 2889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:14.606500 2889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:14.606532 2889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.606559 2889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:14.606575 2889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:14.612085 2889 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:36481
I20260809 08:50:14.612121 3076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:36481 every 8 connection(s)
I20260809 08:50:14.612442 2889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:50:14.617322 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 2889
I20260809 08:50:14.617410 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:50:14.617591 10977 external_mini_cluster.cc:1428] Setting key 95357e133ed1d4f67972ed5590bdc14a
Aug 09 08:50:14 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265414, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:14.621001 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.613102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:59341 (local address 127.10.184.126:39307)
0809 08:50:14.613161 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.613163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.613354 (+ 191us) server_negotiation.cc:407] Connection header received
0809 08:50:14.613819 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.613821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.613833 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.613883 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:14.614749 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.615251 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.616309 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.616529 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.616584 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.618790 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:14.618791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:14.618791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:14.618798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:14.619827 (+ 1029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.620184 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.620185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.620185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.620246 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.620480 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.620480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.620481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.620587 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:14.620637 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.620832 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.620934 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:50:14.621155 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.613079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:59341)
0809 08:50:14.613244 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.613277 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.613722 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.613954 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.613958 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.614482 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:14.614655 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.614659 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.615330 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.615331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.616207 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.616210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.616609 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.616610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.616724 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.617149 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:14.617157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:14.618715 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:14.619889 (+ 1174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.619895 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.619897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.620100 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.620323 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:14.620324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:14.620325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:14.620429 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:14.620628 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:14.620631 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:14.620732 (+ 101us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.620804 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.620974 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
I20260809 08:50:14.621569 3077 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:14.621654 3077 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:14.621780 3077 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
I20260809 08:50:14.622118 2367 ts_manager.cc:194] Registered new tserver with Master: cc0658909f704494abff0400492e39c9 (127.10.184.67:36481)
I20260809 08:50:14.622548 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:59341
I20260809 08:50:14.629273 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:50:14.635329 2432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.630471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43400 (local address 127.10.184.126:39307)
0809 08:50:14.630511 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.630513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.630602 (+ 89us) server_negotiation.cc:407] Connection header received
0809 08:50:14.630655 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.630656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.630664 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.630706 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:14.631167 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.631673 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.632328 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.632495 (+ 167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.632533 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.633546 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:14.633547 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:14.633548 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:14.633552 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:14.634551 (+ 999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.634811 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.634812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.634812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.634852 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:14.634957 (+ 105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:14.634957 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:14.634957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:14.635069 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:14.635134 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.635209 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.635263 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:50:14.636725 2367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:50:14.638300 2367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:50:14.644198 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54020 (local address 127.10.184.65:40763)
0809 08:50:14.640597 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.640602 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.640612 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:50:14.640651 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.640654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.640669 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.640776 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:14.641061 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.641801 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643109 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.643790 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643881 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.643942 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.644077 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.644119 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260809 08:50:14.644208 3090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.0.0.1:54020)
0809 08:50:14.640303 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.640341 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.640402 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.640789 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.640792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.640794 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:14.640854 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.640855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.641889 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.641890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.643017 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.643020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.643894 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.643980 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.643997 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.644057 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.644109 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"thread_start_us":53,"threads_started":1}
I20260809 08:50:14.644301 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43039 (local address 127.0.0.1:39362)
0809 08:50:14.640614 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.640624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.640676 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.640959 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.640960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.640963 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:14.641016 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.641018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.642319 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.642320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.643132 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.643134 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.643996 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.644058 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.644062 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.644108 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.644150 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":2,"thread_start_us":46,"threads_started":1}
I20260809 08:50:14.644375 3091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:36481 (local address 127.0.0.1:56088)
0809 08:50:14.640415 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.640424 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.640462 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.640933 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.640934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.640935 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:14.640967 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.640968 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.642297 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.642298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.643378 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.643381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.644202 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.644202 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.644258 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.644262 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.644289 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.644320 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":25,"thread_start_us":45,"threads_started":1}
I20260809 08:50:14.644368 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.10.184.66:43039)
0809 08:50:14.640797 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.640802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.640811 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:50:14.640845 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.640847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.640861 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.640970 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:14.641101 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.642223 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643200 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.643843 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643908 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.643960 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.644248 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.644291 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260809 08:50:14.644694 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.640429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.10.184.67:36481)
0809 08:50:14.640611 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.640614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.640622 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:50:14.640660 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.640662 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.640675 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.640775 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:14.641104 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.642193 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.643461 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.644105 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.644522 (+ 417us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.644567 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.644586 (+ 19us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.644622 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260809 08:50:14.645109 2771 tablet_service.cc:1467] Processing CreateTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf (DEFAULT_TABLE table=test-table [id=0b7348246a754fd59702a0cf60c8f8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:14.645496 2771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f70c7fc0006425684822dd5b4fb17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:14.645447 2982 tablet_service.cc:1467] Processing CreateTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf (DEFAULT_TABLE table=test-table [id=0b7348246a754fd59702a0cf60c8f8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:14.645772 2982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f70c7fc0006425684822dd5b4fb17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:14.645777 2574 tablet_service.cc:1467] Processing CreateTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf (DEFAULT_TABLE table=test-table [id=0b7348246a754fd59702a0cf60c8f8da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:14.646080 2574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f70c7fc0006425684822dd5b4fb17cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:14.648089 3097 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Bootstrap starting.
I20260809 08:50:14.648638 3097 tablet_bootstrap.cc:654] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:14.648919 3099 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Bootstrap starting.
I20260809 08:50:14.649320 3101 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Bootstrap starting.
I20260809 08:50:14.649516 3097 log.cc:824] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:14.649551 3099 tablet_bootstrap.cc:654] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:14.649914 3101 tablet_bootstrap.cc:654] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:14.650051 3099 log.cc:824] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:14.650338 3097 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: No bootstrap required, opened a new log
I20260809 08:50:14.650396 3097 ts_tablet_manager.cc:1397] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:50:14.650437 3101 log.cc:824] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:14.650760 3099 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: No bootstrap required, opened a new log
I20260809 08:50:14.650818 3099 ts_tablet_manager.cc:1397] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:50:14.651146 3101 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: No bootstrap required, opened a new log
I20260809 08:50:14.651217 3101 ts_tablet_manager.cc:1397] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:50:14.652307 3099 raft_consensus.cc:348] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.652271 3097 raft_consensus.cc:348] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.652438 3099 raft_consensus.cc:374] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:14.652443 3097 raft_consensus.cc:374] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:14.652470 3099 raft_consensus.cc:729] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0658909f704494abff0400492e39c9, State: Initialized, Role: FOLLOWER
I20260809 08:50:14.652503 3097 raft_consensus.cc:729] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9a96c37e0a49cdb987a1eeeccf1f15, State: Initialized, Role: FOLLOWER
I20260809 08:50:14.652582 3099 consensus_queue.cc:260] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.652648 3097 consensus_queue.cc:260] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.652853 3099 ts_tablet_manager.cc:1428] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:50:14.652942 3097 ts_tablet_manager.cc:1428] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:50:14.652853 3077 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
I20260809 08:50:14.653062 2881 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
I20260809 08:50:14.653092 3101 raft_consensus.cc:348] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.653219 3101 raft_consensus.cc:374] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:14.653249 3101 raft_consensus.cc:729] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf79186ef85a474abb1556f20cf23d62, State: Initialized, Role: FOLLOWER
I20260809 08:50:14.653373 3101 consensus_queue.cc:260] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.653607 3101 ts_tablet_manager.cc:1428] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:50:14.653636 2667 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
I20260809 08:50:14.709070 3107 raft_consensus.cc:482] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:14.709218 3107 raft_consensus.cc:504] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.709568 3107 leader_election.cc:288] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039), cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:14.712172 3079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.709727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43039 (local address 127.10.184.67:60031)
0809 08:50:14.709794 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.709807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.709856 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.710020 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.710022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.710024 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:14.710089 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.710091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.710688 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.710689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.711406 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.711407 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.711935 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.711936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.712004 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.712007 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.712051 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.712092 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 08:50:14.712163 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.709840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:60031 (local address 127.10.184.66:43039)
0809 08:50:14.709886 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.709888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.709898 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:50:14.709928 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.709929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.709940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.710001 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:14.710148 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.710611 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.711488 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.711862 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.711924 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.711933 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.712063 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.712099 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:50:14.712260 3092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.709917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:52891 (local address 127.10.184.65:40763)
0809 08:50:14.709983 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.709985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.710009 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:50:14.710036 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.710037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.710054 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.710094 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:14.710246 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.710696 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.711581 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.711952 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.712014 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.712023 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.712159 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.712198 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:50:14.712271 3110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.709730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.10.184.67:52891)
0809 08:50:14.709932 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.709940 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:14.709981 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:14.710109 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:14.710110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:14.710111 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:14.710184 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.710186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.710772 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.710773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.711497 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:14.711498 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.712030 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.712031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:14.712096 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.712099 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:50:14.712143 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:50:14.712181 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":13,"thread_start_us":111,"threads_started":1}
I20260809 08:50:14.712527 2804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f70c7fc0006425684822dd5b4fb17cf" candidate_uuid: "cc0658909f704494abff0400492e39c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" is_pre_election: true
I20260809 08:50:14.712575 2606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f70c7fc0006425684822dd5b4fb17cf" candidate_uuid: "cc0658909f704494abff0400492e39c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79186ef85a474abb1556f20cf23d62" is_pre_election: true
I20260809 08:50:14.712687 2804 raft_consensus.cc:2454] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0658909f704494abff0400492e39c9 in term 0.
I20260809 08:50:14.712705 2606 raft_consensus.cc:2454] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc0658909f704494abff0400492e39c9 in term 0.
I20260809 08:50:14.712944 2914 leader_election.cc:302] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc0658909f704494abff0400492e39c9, cf79186ef85a474abb1556f20cf23d62; no voters:
I20260809 08:50:14.713106 3107 raft_consensus.cc:2781] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:50:14.713169 3107 raft_consensus.cc:482] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:50:14.713199 3107 raft_consensus.cc:3037] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:14.713887 3107 raft_consensus.cc:504] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.714012 3107 leader_election.cc:288] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [CANDIDATE]: Term 1 election: Requested vote from peers 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039), cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:14.714156 2804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f70c7fc0006425684822dd5b4fb17cf" candidate_uuid: "cc0658909f704494abff0400492e39c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
I20260809 08:50:14.714172 2606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f70c7fc0006425684822dd5b4fb17cf" candidate_uuid: "cc0658909f704494abff0400492e39c9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79186ef85a474abb1556f20cf23d62"
I20260809 08:50:14.714237 2606 raft_consensus.cc:3037] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:14.714234 2804 raft_consensus.cc:3037] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:14.714916 2606 raft_consensus.cc:2454] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0658909f704494abff0400492e39c9 in term 1.
I20260809 08:50:14.715054 2804 raft_consensus.cc:2454] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc0658909f704494abff0400492e39c9 in term 1.
I20260809 08:50:14.715075 2914 leader_election.cc:302] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc0658909f704494abff0400492e39c9, cf79186ef85a474abb1556f20cf23d62; no voters:
I20260809 08:50:14.715176 3107 raft_consensus.cc:2781] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:50:14.715329 3107 raft_consensus.cc:686] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 1 LEADER]: Becoming Leader. State: Replica: cc0658909f704494abff0400492e39c9, State: Running, Role: LEADER
I20260809 08:50:14.715425 3107 consensus_queue.cc:237] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:14.716236 2365 catalog_manager.cc:5495] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 reported cstate change: term changed from 0 to 1, leader changed from <none> to cc0658909f704494abff0400492e39c9 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "cc0658909f704494abff0400492e39c9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } health_report { overall_health: UNKNOWN } } }
W20260809 08:50:14.718325 2882 tablet.cc:2369] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:50:14.738258 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.735921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56100 (local address 127.10.184.67:36481)
0809 08:50:14.735978 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:14.735980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:14.735990 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:50:14.736030 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:14.736031 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:14.736049 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:14.736111 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:50:14.736223 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.736701 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.737468 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:14.737654 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:14.737711 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:14.737874 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:50:14.737974 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:50:14.738055 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:14.738125 (+ 70us) server_negotiation.cc:299] Negotiation successful
0809 08:50:14.738150 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:50:14.740428 2804 raft_consensus.cc:1261] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Refusing update from remote peer cc0658909f704494abff0400492e39c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:50:14.740428 2606 raft_consensus.cc:1261] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Refusing update from remote peer cc0658909f704494abff0400492e39c9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:50:14.740751 3112 consensus_queue.cc:1035] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:50:14.740883 3107 consensus_queue.cc:1035] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:50:14.744174 3117 mvcc.cc:205] Tried to move back new op lower bound from 7316543138774532096 to 7316543138675044352. Current Snapshot: MvccSnapshot[applied={T|T < 7316543138774532096 or (T in {7316543138774532096})}]
I20260809 08:50:14.744341 3118 mvcc.cc:205] Tried to move back new op lower bound from 7316543138774532096 to 7316543138675044352. Current Snapshot: MvccSnapshot[applied={T|T < 7316543138774532096}]
I20260809 08:50:14.745565 2365 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:50:14.745649 2365 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:50:14.746366 2365 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Sending DeleteTablet for 3 replicas of tablet 6f70c7fc0006425684822dd5b4fb17cf
I20260809 08:50:14.746697 2982 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:14 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:56088
I20260809 08:50:14.746698 2574 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:14 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:54020
I20260809 08:50:14.746914 2771 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:14 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:39362
I20260809 08:50:14.747131 3131 tablet_replica.cc:330] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: stopping tablet replica
I20260809 08:50:14.747215 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 2466
I20260809 08:50:14.747248 3130 tablet_replica.cc:330] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: stopping tablet replica
I20260809 08:50:14.747259 3131 raft_consensus.cc:2229] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:50:14.747329 3130 raft_consensus.cc:2229] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:50:14.747424 3130 raft_consensus.cc:2258] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:14.747418 3131 raft_consensus.cc:2258] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:14.749770 3129 tablet_replica.cc:330] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: stopping tablet replica
I20260809 08:50:14.749889 3129 raft_consensus.cc:2229] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:50:14.750020 3129 raft_consensus.cc:2258] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:14.751242 3131 ts_tablet_manager.cc:1905] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:14.752867 3129 ts_tablet_manager.cc:1905] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:14.752890 3131 ts_tablet_manager.cc:1918] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:50:14.753026 3131 log.cc:1196] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/6f70c7fc0006425684822dd5b4fb17cf
I20260809 08:50:14.753327 3131 ts_tablet_manager.cc:1939] T 6f70c7fc0006425684822dd5b4fb17cf P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting consensus metadata
I20260809 08:50:14.753772 2337 catalog_manager.cc:4841] TS 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): tablet 6f70c7fc0006425684822dd5b4fb17cf (table test-table [id=0b7348246a754fd59702a0cf60c8f8da]) successfully deleted
I20260809 08:50:14.754769 3129 ts_tablet_manager.cc:1918] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:50:14.754865 3129 log.cc:1196] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/6f70c7fc0006425684822dd5b4fb17cf
I20260809 08:50:14.755337 3129 ts_tablet_manager.cc:1939] T 6f70c7fc0006425684822dd5b4fb17cf P cc0658909f704494abff0400492e39c9: Deleting consensus metadata
I20260809 08:50:14.755944 2336 catalog_manager.cc:4841] TS cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): tablet 6f70c7fc0006425684822dd5b4fb17cf (table test-table [id=0b7348246a754fd59702a0cf60c8f8da]) successfully deleted
W20260809 08:50:14.775657 2340 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:40763 (error 108)
I20260809 08:50:14.775980 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 2675
I20260809 08:50:14.776139 3089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:14.776014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.0.0.1:54026)
0809 08:50:14.776069 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:14.776106 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 08:50:14.783340 2340 catalog_manager.cc:4584] TS cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f70c7fc0006425684822dd5b4fb17cf: Network error: Client connection negotiation failed: client connection to 127.10.184.65:40763: connect: Connection refused (error 111)
I20260809 08:50:14.790449 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 2889
I20260809 08:50:14.798243 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 2309
I20260809 08:50:34.811236 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--webserver_interface=127.10.184.126
--webserver_port=42127
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:34.893414 3134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:34.893608 3134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:34.893636 3134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:34.894735 3134 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:50:34.894773 3134 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:50:34.894790 3134 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:50:34.894804 3134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:50:34.894820 3134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:34.894838 3134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:34.894861 3134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:50:34.894878 3134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:50:34.896338 3134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=42127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:34.896564 3134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:34.896806 3134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:34.905706 3143 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:34.905714 3140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:34.905930 3141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:34.908000 3134 server_base.cc:1034] running on GCE node
I20260809 08:50:34.908258 3134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:34.908560 3134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:34.909698 3134 hybrid_clock.cc:648] HybridClock initialized: now 1786265434909676 us; error 34 us; skew 500 ppm
Aug 09 08:50:34 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:34.912062 3134 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:50:34.912518 3134 webserver.cc:460] Webserver started at http://127.10.184.126:42127/ using document root <none> and password file <none>
I20260809 08:50:34.912745 3134 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:34.912809 3134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:34.914179 3134 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:50:34.914790 3155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:34.914950 3134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:50:34.915052 3134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9ea9b01e867448cf8647a344dd982099"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "c7ebb6cd8c69aa3bdcf6482aff67b61d"
server_key_iv: "4e757ccf1f008e63b2c147e0236fd35b"
server_key_version: "encryptionkey@0"
I20260809 08:50:34.915439 3134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:34.939675 3134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:34.941442 3134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:34.941576 3134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:34.945547 3134 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39307
I20260809 08:50:34.945591 3233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39307 every 8 connection(s)
I20260809 08:50:34.945950 3134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:50:34.947109 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 3134
I20260809 08:50:34.947365 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:40763
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=35211
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:50:34.947494 3234 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:50:34.948156 3234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap starting.
I20260809 08:50:34.956609 3234 log.cc:824] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:34.958465 3234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:50:34.958731 3234 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap complete.
I20260809 08:50:34.960800 3234 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:34.961058 3234 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Initialized, Role: FOLLOWER
I20260809 08:50:34.961225 3234 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:34.961300 3234 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:50:34.961335 3234 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:50:34.961382 3234 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 1 FOLLOWER]: Advancing to term 2
I20260809 08:50:34.962352 3234 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:34.962441 3234 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea9b01e867448cf8647a344dd982099; no voters:
I20260809 08:50:34.962584 3234 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 08:50:34.962639 3240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 08:50:34.962785 3240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 LEADER]: Becoming Leader. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Running, Role: LEADER
I20260809 08:50:34.962826 3234 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:50:34.962878 3240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:34.963114 3240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea9b01e867448cf8647a344dd982099. Latest consensus state: current_term: 2 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:34.963186 3240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:34.963311 3244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:34.963363 3244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:34.963634 3249 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:50:34.964377 3249 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:50:34.964476 3249 catalog_manager.cc:1259] Loaded cluster ID: f10d41a1a30e40fda7a3988a3d3c81f4
I20260809 08:50:34.964504 3249 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:50:34.965724 3249 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:50:34.965914 3249 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Loaded TSK: 0
I20260809 08:50:34.977105 3249 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Generated new TSK 1
I20260809 08:50:34.977195 3249 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:50:35.032578 3237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:35.032765 3237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:35.032786 3237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:35.033908 3237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:35.033949 3237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:35.033973 3237 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:50:35.035382 3237 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:40763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=35211
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:35.035611 3237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:35.035816 3237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:35.040482 3276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:35.040514 3278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:35.040664 3237 server_base.cc:1034] running on GCE node
W20260809 08:50:35.040719 3280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:35.040927 3237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:35.041169 3237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:35.042312 3237 hybrid_clock.cc:648] HybridClock initialized: now 1786265435042295 us; error 32 us; skew 500 ppm
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:35.044813 3237 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:50:35.045277 3237 webserver.cc:460] Webserver started at http://127.10.184.65:35211/ using document root <none> and password file <none>
I20260809 08:50:35.045496 3237 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:35.045557 3237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:35.046813 3237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:35.047441 3294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.047644 3237 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:35.047744 3237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "cf79186ef85a474abb1556f20cf23d62"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "fe165eb42bfbe0c945634f60cfddc4e7"
server_key_iv: "ee90a1bbd35fc113f5c2496b3863af09"
server_key_version: "encryptionkey@0"
I20260809 08:50:35.048122 3237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:35.079649 3237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:35.081774 3237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:35.081894 3237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:35.082109 3237 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:35.082576 3306 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:50:35.084138 3237 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:50:35.084187 3237 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:50:35.084216 3237 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:50:35.084915 3237 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:50:35.084964 3237 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:35.085059 3306 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Bootstrap starting.
I20260809 08:50:35.092350 3237 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:40763
I20260809 08:50:35.092406 3463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:40763 every 8 connection(s)
I20260809 08:50:35.092856 3237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:50:35.096057 3306 log.cc:824] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:35.098650 3306 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:50:35.098958 3306 tablet_bootstrap.cc:492] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Bootstrap complete.
I20260809 08:50:35.099980 3306 ts_tablet_manager.cc:1397] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:35.100876 3306 raft_consensus.cc:348] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.101153 3306 raft_consensus.cc:729] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf79186ef85a474abb1556f20cf23d62, State: Initialized, Role: FOLLOWER
I20260809 08:50:35.101289 3306 consensus_queue.cc:260] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.101539 3306 ts_tablet_manager.cc:1428] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Time spent starting tablet: real 0.002s user 0.003s sys 0.001s
I20260809 08:50:35.103016 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 3237
I20260809 08:50:35.103288 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:43039
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=45329
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:50:35.104030 3466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.093526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:43653)
0809 08:50:35.093703 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.093760 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.094375 (+ 615us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.094822 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.094827 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.095197 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:35.095541 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.095548 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.096922 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.096923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.098155 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.098158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.098737 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.098738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.098878 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.099357 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:35.099366 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:35.100796 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:35.102520 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.102527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.102529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.102733 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.103037 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.103038 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.103038 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.103167 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.103585 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:35.103591 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:35.103702 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.103779 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.103892 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 08:50:35.104168 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.093666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:43653 (local address 127.10.184.126:39307)
0809 08:50:35.093819 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.093828 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.093861 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:50:35.094603 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.094608 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.094650 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.094751 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.095660 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.096766 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.098233 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.098627 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.098705 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.100887 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.100907 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.100908 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.100920 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.102444 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.102806 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.102809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.102809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.102956 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.103222 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.103223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.103223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.103511 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.103590 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.103851 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.103996 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":79,"threads_started":1}
I20260809 08:50:35.105083 3464 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:35.105162 3464 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:35.105305 3464 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
I20260809 08:50:35.105979 3180 ts_manager.cc:194] Registered new tserver with Master: cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:35.106990 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:43653
I20260809 08:50:35.107847 3464 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265434, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.65@KRBTEST.COM
I20260809 08:50:35.114117 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.106585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33940 (local address 127.10.184.65:40763)
0809 08:50:35.106708 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.106711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.106720 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:50:35.106798 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.106800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.106812 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.106865 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.107413 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.108331 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.109360 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.109634 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.109682 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.111604 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.111611 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.111612 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.111618 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.113023 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.113361 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.113364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.113364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.113421 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.113582 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.113583 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.113583 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.113722 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.113790 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.113953 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.114032 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":72,"threads_started":1}
I20260809 08:50:35.114121 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.106495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.0.0.1:33940)
0809 08:50:35.106634 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.106667 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.106728 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.106884 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.106886 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.107175 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:35.107224 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.107226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.108409 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.108410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.109279 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.109282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.109699 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.109700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.109780 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.110168 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:35.110174 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:35.111521 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:35.113088 (+ 1567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.113095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.113097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.113289 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.113476 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.113477 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.113478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.113533 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.113785 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:35.113792 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:35.113841 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.113912 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.114026 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":75,"threads_started":1}
I20260809 08:50:35.114451 3368 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f70c7fc0006425684822dd5b4fb17cf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:14 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:33940
I20260809 08:50:35.114861 3481 tablet_replica.cc:330] stopping tablet replica
I20260809 08:50:35.114969 3481 raft_consensus.cc:2229] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:50:35.115036 3481 raft_consensus.cc:2258] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:35.115325 3481 ts_tablet_manager.cc:1905] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:35.117587 3481 ts_tablet_manager.cc:1918] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:50:35.117660 3481 log.cc:1196] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal/wals/6f70c7fc0006425684822dd5b4fb17cf
I20260809 08:50:35.117844 3481 ts_tablet_manager.cc:1939] T 6f70c7fc0006425684822dd5b4fb17cf P cf79186ef85a474abb1556f20cf23d62: Deleting consensus metadata
I20260809 08:50:35.118312 3159 catalog_manager.cc:4841] TS cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763): tablet 6f70c7fc0006425684822dd5b4fb17cf (table test-table [id=0b7348246a754fd59702a0cf60c8f8da]) successfully deleted
W20260809 08:50:35.186009 3472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:35.186193 3472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:35.186214 3472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:35.187278 3472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:35.187309 3472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:35.187335 3472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:50:35.188704 3472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:43039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=45329
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:35.188922 3472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:35.189133 3472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:35.194012 3490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:35.194024 3492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:35.194067 3494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:35.194130 3472 server_base.cc:1034] running on GCE node
I20260809 08:50:35.194414 3472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:35.194653 3472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:35.195823 3472 hybrid_clock.cc:648] HybridClock initialized: now 1786265435195804 us; error 36 us; skew 500 ppm
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:35.198186 3472 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:50:35.198576 3472 webserver.cc:460] Webserver started at http://127.10.184.66:45329/ using document root <none> and password file <none>
I20260809 08:50:35.198784 3472 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:35.198849 3472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:35.200078 3472 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:35.200600 3503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.200738 3472 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:35.200824 3472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "9e3c64bbe89f698314820fb33fbd1038"
server_key_iv: "60218ca3da604eb6ca6ace22c2693957"
server_key_version: "encryptionkey@0"
I20260809 08:50:35.201154 3472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:35.219619 3472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:35.221760 3472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:35.221895 3472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:35.222103 3472 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:35.222406 3472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:35.222441 3472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.222471 3472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:35.222488 3472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.228014 3472 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43039
I20260809 08:50:35.228070 3678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43039 every 8 connection(s)
I20260809 08:50:35.228451 3472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:50:35.230083 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 3472
I20260809 08:50:35.230327 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:36481
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=40925
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:35.237654 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.229144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:44047 (local address 127.10.184.126:39307)
0809 08:50:35.229214 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.229216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.229404 (+ 188us) server_negotiation.cc:407] Connection header received
0809 08:50:35.229868 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.229869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.229879 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.229938 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.230980 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.231479 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.232457 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.232669 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.232725 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.234986 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.234987 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.234987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.234994 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.236213 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.236680 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.236681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.236681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.236729 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.237017 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.237018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.237018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.237137 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.237195 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.237468 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.237561 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:50:35.237787 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.229136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:44047)
0809 08:50:35.229296 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.229331 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.229780 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.230054 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.230057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.230699 (+ 642us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:35.230883 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.230889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.231560 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.231561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.232375 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.232377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.232740 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.232741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.232871 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.233330 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:35.233339 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:35.234900 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:35.236292 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.236299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.236301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.236592 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.236783 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.236784 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.236784 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.236951 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.237207 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:35.237215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:35.237342 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.237434 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.237582 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260809 08:50:35.238227 3679 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:35.238330 3679 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:35.238517 3679 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
I20260809 08:50:35.238909 3180 ts_manager.cc:194] Registered new tserver with Master: 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
I20260809 08:50:35.239260 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:44047
W20260809 08:50:35.311012 3682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:35.311197 3682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:35.311218 3682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:35.312318 3682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:35.312356 3682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:35.312384 3682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:50:35.313792 3682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:36481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=40925
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:35.313994 3682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:35.314196 3682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:35.319634 3695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:35.319679 3690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:35.319674 3691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:35.319962 3682 server_base.cc:1034] running on GCE node
I20260809 08:50:35.320156 3682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:35.320393 3682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:35.321532 3682 hybrid_clock.cc:648] HybridClock initialized: now 1786265435321520 us; error 27 us; skew 500 ppm
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:35.323810 3682 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:50:35.324199 3682 webserver.cc:460] Webserver started at http://127.10.184.67:40925/ using document root <none> and password file <none>
I20260809 08:50:35.324410 3682 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:35.324471 3682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:35.325778 3682 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:35.326310 3704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.326452 3682 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:35.326530 3682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "cc0658909f704494abff0400492e39c9"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "bf1f543914fbfedc5358c77fba97eb60"
server_key_iv: "0af70fbdad39d947e8c0cc854538fb89"
server_key_version: "encryptionkey@0"
I20260809 08:50:35.326833 3682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:35.349081 3682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:35.351121 3682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:35.351258 3682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:35.351467 3682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:35.351760 3682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:35.351805 3682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.351833 3682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:35.351856 3682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:35.357470 3682 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:36481
I20260809 08:50:35.357537 3870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:36481 every 8 connection(s)
I20260809 08:50:35.357815 3682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:50:35 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265435, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:50:35.365599 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 3682
I20260809 08:50:35.366232 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.358463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:57951 (local address 127.10.184.126:39307)
0809 08:50:35.358521 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.358523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.358701 (+ 178us) server_negotiation.cc:407] Connection header received
0809 08:50:35.359189 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.359190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.359203 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.359254 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.360144 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.360628 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.361633 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.361841 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.361896 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.363901 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.363902 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.363902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.363909 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.364999 (+ 1090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.365336 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.365337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.365337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.365382 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.365618 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.365618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.365618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.365729 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.365776 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.366057 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.366145 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:50:35.366379 3876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.358420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:57951)
0809 08:50:35.358586 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.358619 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.359090 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.359322 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.359326 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.359865 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:35.360043 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.360048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.360707 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.360708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.361552 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.361555 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.361908 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.361909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.362021 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.362479 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:35.362487 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:35.363814 (+ 1327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:35.365060 (+ 1246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.365066 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.365068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.365280 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.365429 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:35.365430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:35.365430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:35.365561 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:35.365775 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:35.365782 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:35.365917 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.366022 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.366183 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260809 08:50:35.366804 3871 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39307
I20260809 08:50:35.366905 3871 heartbeater.cc:464] Registering TS with master...
I20260809 08:50:35.367084 3871 heartbeater.cc:511] Master 127.10.184.126:39307 requested a full tablet report, sending...
I20260809 08:50:35.367465 3180 ts_manager.cc:194] Registered new tserver with Master: cc0658909f704494abff0400492e39c9 (127.10.184.67:36481)
I20260809 08:50:35.367920 3180 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:57951
I20260809 08:50:35.372340 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.365981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.10.184.126:39307)
0809 08:50:35.366096 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.366098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.366190 (+ 92us) server_negotiation.cc:407] Connection header received
0809 08:50:35.366248 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.366249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.366260 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.366317 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.367182 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.367860 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.368862 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.369106 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.369156 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.370121 (+ 965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.370122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.370122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.370128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.371352 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.371601 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.371602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.371602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.371659 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.371850 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.371851 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.371851 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.371971 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.372139 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.372177 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.372254 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":54,"threads_started":1}
I20260809 08:50:35.373061 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:50:35.375430 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.373327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59238 (local address 127.10.184.126:39307)
0809 08:50:35.373362 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.373364 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.373472 (+ 108us) server_negotiation.cc:407] Connection header received
0809 08:50:35.373539 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.373540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.373565 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.373614 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:50:35.373708 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.374130 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.374835 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.375015 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.375065 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.375212 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:50:35.375250 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 08:50:35.375378 (+ 128us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59238: token expired
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 08:50:35.375479 3878 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59238: token expired
W20260809 08:50:35.375552 3083 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.10.184.126:39307)
I20260809 08:50:35.380815 3878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.375774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59254 (local address 127.10.184.126:39307)
0809 08:50:35.375804 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.375806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.375813 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:50:35.375865 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.375866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.375874 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.375912 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:35.376340 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.376770 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.377479 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.377652 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.377710 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.378680 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:35.378682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:35.378682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:35.378688 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:35.379876 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.380164 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.380165 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.380165 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.380215 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:35.380371 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:35.380372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:35.380372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:35.380468 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:35.380545 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.380672 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.380738 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:50:35.381752 3180 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59254:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:50:35.383256 3180 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:50:35.385135 3368 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4eec643b684e579575e67beff9a9e8 (DEFAULT_TABLE table=test-table [id=6d2541c63bba4747a538909e368010a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:35.385432 3368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4eec643b684e579575e67beff9a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:35.386950 3306 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Bootstrap starting.
I20260809 08:50:35.387283 3306 tablet_bootstrap.cc:654] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:35.387846 3306 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: No bootstrap required, opened a new log
I20260809 08:50:35.387897 3306 ts_tablet_manager.cc:1397] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 08:50:35.388230 3306 raft_consensus.cc:348] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.388293 3306 raft_consensus.cc:374] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:35.388329 3306 raft_consensus.cc:729] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf79186ef85a474abb1556f20cf23d62, State: Initialized, Role: FOLLOWER
I20260809 08:50:35.388381 3306 consensus_queue.cc:260] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.388412 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.384870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:36481 (local address 127.0.0.1:41154)
0809 08:50:35.384930 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.384939 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.384975 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.385341 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.385342 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.385345 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:35.385391 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.385392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.386382 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.386384 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.387453 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.387456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.388156 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.388157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.388239 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.388260 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.388308 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.388348 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 08:50:35.388434 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.384938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.10.184.67:36481)
0809 08:50:35.385084 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.385088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.385097 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:50:35.385131 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.385133 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.385160 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.385242 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:35.385462 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.386309 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.387520 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.388080 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.388153 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.388207 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.388328 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.388364 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260809 08:50:35.388470 3306 ts_tablet_manager.cc:1428] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 08:50:35.389243 3773 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4eec643b684e579575e67beff9a9e8 (DEFAULT_TABLE table=test-table [id=6d2541c63bba4747a538909e368010a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:35.389559 3773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4eec643b684e579575e67beff9a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:35.390512 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.384918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43039 (local address 127.0.0.1:42440)
0809 08:50:35.386993 (+ 2075us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.387003 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.387060 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.387267 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.387268 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.387271 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:35.387314 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.387316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.388581 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.388582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.389639 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.389642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.390257 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.390258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.390320 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.390324 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.390409 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.390448 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2051,"thread_start_us":87,"threads_started":1}
I20260809 08:50:35.390540 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.385144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42440 (local address 127.10.184.66:43039)
0809 08:50:35.385339 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.385343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.387113 (+ 1770us) server_negotiation.cc:407] Connection header received
0809 08:50:35.387160 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.387163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.387179 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.387251 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:35.387383 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.388409 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.389721 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.390172 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.390230 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.390265 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.390424 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.390463 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":85,"threads_started":1}
I20260809 08:50:35.391382 3575 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4eec643b684e579575e67beff9a9e8 (DEFAULT_TABLE table=test-table [id=6d2541c63bba4747a538909e368010a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:50:35.391726 3575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4eec643b684e579575e67beff9a9e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:50:35.392782 3888 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Bootstrap starting.
I20260809 08:50:35.393363 3888 tablet_bootstrap.cc:654] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:35.393885 3888 log.cc:824] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:35.394582 3888 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: No bootstrap required, opened a new log
I20260809 08:50:35.394635 3888 ts_tablet_manager.cc:1397] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:50:35.394785 3892 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Bootstrap starting.
I20260809 08:50:35.395453 3892 tablet_bootstrap.cc:654] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Neither blocks nor log segments found. Creating new log.
I20260809 08:50:35.395998 3892 log.cc:824] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:35.396497 3888 raft_consensus.cc:348] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.396672 3888 raft_consensus.cc:374] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:35.396723 3888 raft_consensus.cc:729] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc0658909f704494abff0400492e39c9, State: Initialized, Role: FOLLOWER
I20260809 08:50:35.396733 3892 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: No bootstrap required, opened a new log
I20260809 08:50:35.396788 3892 ts_tablet_manager.cc:1397] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:50:35.396839 3888 consensus_queue.cc:260] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.397143 3888 ts_tablet_manager.cc:1428] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:50:35.397176 3871 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
I20260809 08:50:35.398427 3892 raft_consensus.cc:348] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.398561 3892 raft_consensus.cc:374] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:50:35.398581 3892 raft_consensus.cc:729] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c9a96c37e0a49cdb987a1eeeccf1f15, State: Initialized, Role: FOLLOWER
I20260809 08:50:35.398655 3892 consensus_queue.cc:260] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.398854 3892 ts_tablet_manager.cc:1428] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:50:35.398916 3679 heartbeater.cc:503] Master 127.10.184.126:39307 was elected leader, sending a full tablet report...
I20260809 08:50:35.414055 3898 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:35.414122 3898 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.414367 3898 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:35.416993 3882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.414574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:56297 (local address 127.10.184.67:36481)
0809 08:50:35.414621 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.414623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.414631 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:50:35.414661 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.414661 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.414670 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.414722 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:35.414850 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.415338 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.416251 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.416696 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.416745 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.416751 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.416925 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.416951 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:50:35.417019 3681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.414512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:36481 (local address 127.10.184.66:56297)
0809 08:50:35.414549 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.414559 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.414598 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.414734 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.414735 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.414736 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:35.414791 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.414792 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.415405 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.415406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.416178 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.416181 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.416761 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.416762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.416839 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.416842 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.416918 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.416954 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 08:50:35.417413 3899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.414519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.10.184.66:42621)
0809 08:50:35.414689 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.414697 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:35.414762 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:35.414915 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:35.414915 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:35.414917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:50:35.414980 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.414981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.415661 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.415661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.416441 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:35.416443 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.417204 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.417205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:35.417257 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.417260 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:50:35.417307 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:50:35.417341 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":37,"thread_start_us":79,"threads_started":1}
I20260809 08:50:35.417423 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.414622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:42621 (local address 127.10.184.65:40763)
0809 08:50:35.414661 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.414675 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.414739 (+ 64us) server_negotiation.cc:407] Connection header received
0809 08:50:35.414826 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.414827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.414848 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.414905 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:50:35.415042 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.415583 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.416517 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.417014 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.417084 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.417140 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.417325 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.417366 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:50:35.417410 3804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4eec643b684e579575e67beff9a9e8" candidate_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0658909f704494abff0400492e39c9" is_pre_election: true
I20260809 08:50:35.417515 3804 raft_consensus.cc:2454] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9a96c37e0a49cdb987a1eeeccf1f15 in term 0.
I20260809 08:50:35.417742 3401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4eec643b684e579575e67beff9a9e8" candidate_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79186ef85a474abb1556f20cf23d62" is_pre_election: true
I20260809 08:50:35.417861 3401 raft_consensus.cc:2454] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c9a96c37e0a49cdb987a1eeeccf1f15 in term 0.
I20260809 08:50:35.418015 3504 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9; no voters:
I20260809 08:50:35.418150 3898 raft_consensus.cc:2781] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:50:35.418220 3898 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:50:35.418242 3898 raft_consensus.cc:3037] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:35.418807 3898 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.418915 3898 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [CANDIDATE]: Term 1 election: Requested vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763)
I20260809 08:50:35.419057 3401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4eec643b684e579575e67beff9a9e8" candidate_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cf79186ef85a474abb1556f20cf23d62"
I20260809 08:50:35.419077 3804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4eec643b684e579575e67beff9a9e8" candidate_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc0658909f704494abff0400492e39c9"
I20260809 08:50:35.419131 3401 raft_consensus.cc:3037] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:35.419138 3804 raft_consensus.cc:3037] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:50:35.419752 3804 raft_consensus.cc:2454] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9a96c37e0a49cdb987a1eeeccf1f15 in term 1.
I20260809 08:50:35.419778 3401 raft_consensus.cc:2454] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c9a96c37e0a49cdb987a1eeeccf1f15 in term 1.
I20260809 08:50:35.419898 3504 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9; no voters:
I20260809 08:50:35.419994 3898 raft_consensus.cc:2781] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:50:35.420133 3898 raft_consensus.cc:686] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 LEADER]: Becoming Leader. State: Replica: 8c9a96c37e0a49cdb987a1eeeccf1f15, State: Running, Role: LEADER
I20260809 08:50:35.420215 3898 consensus_queue.cc:237] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:35.420876 3180 catalog_manager.cc:5495] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66). New cstate: current_term: 1 leader_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } health_report { overall_health: UNKNOWN } } }
I20260809 08:50:35.433871 3883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.431762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.10.184.66:43039)
0809 08:50:35.431806 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:35.431807 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:35.431815 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:50:35.431862 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:35.431863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:35.431879 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:35.431937 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:50:35.432035 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.432482 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.433189 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:35.433384 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:35.433435 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:35.433571 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:50:35.433672 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:50:35.433737 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:35.433772 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 08:50:35.433790 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:50:35.435371 3804 raft_consensus.cc:1261] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Refusing update from remote peer 8c9a96c37e0a49cdb987a1eeeccf1f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:50:35.435379 3401 raft_consensus.cc:1261] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Refusing update from remote peer 8c9a96c37e0a49cdb987a1eeeccf1f15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:50:35.435683 3901 consensus_queue.cc:1035] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:50:35.435792 3898 consensus_queue.cc:1035] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:50:35.438257 3905 mvcc.cc:205] Tried to move back new op lower bound from 7316543223541178368 to 7316543223481753600. Current Snapshot: MvccSnapshot[applied={T|T < 7316543223541178368 or (T in {7316543223541178368})}]
I20260809 08:50:35.438851 3906 mvcc.cc:205] Tried to move back new op lower bound from 7316543223541178368 to 7316543223481753600. Current Snapshot: MvccSnapshot[applied={T|T < 7316543223541178368 or (T in {7316543223541178368})}]
I20260809 08:50:35.440119 3180 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:50:35.440196 3180 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59254:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:50:35.440829 3180 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Sending DeleteTablet for 3 replicas of tablet 2b4eec643b684e579575e67beff9a9e8
I20260809 08:50:35.441131 3575 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4eec643b684e579575e67beff9a9e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:35 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:42440
I20260809 08:50:35.441120 3773 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4eec643b684e579575e67beff9a9e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:35 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41154
I20260809 08:50:35.441188 3368 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4eec643b684e579575e67beff9a9e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:50:35 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:33940
I20260809 08:50:35.441262 3481 tablet_replica.cc:330] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: stopping tablet replica
I20260809 08:50:35.441303 3481 raft_consensus.cc:2229] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:50:35.441359 3481 raft_consensus.cc:2258] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:35.441583 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 3237
I20260809 08:50:35.441633 3481 ts_tablet_manager.cc:1905] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:35.452773 3916 tablet_replica.cc:330] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: stopping tablet replica
I20260809 08:50:35.452884 3916 raft_consensus.cc:2229] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:50:35.452983 3916 raft_consensus.cc:2258] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:35.454080 3916 ts_tablet_manager.cc:1905] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:35.455502 3918 tablet_replica.cc:330] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: stopping tablet replica
I20260809 08:50:35.455576 3916 ts_tablet_manager.cc:1918] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:50:35.455600 3918 raft_consensus.cc:2229] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:50:35.455642 3916 log.cc:1196] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/2b4eec643b684e579575e67beff9a9e8
I20260809 08:50:35.455722 3918 raft_consensus.cc:2258] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:50:35.455910 3916 ts_tablet_manager.cc:1939] T 2b4eec643b684e579575e67beff9a9e8 P 8c9a96c37e0a49cdb987a1eeeccf1f15: Deleting consensus metadata
I20260809 08:50:35.456341 3157 catalog_manager.cc:4841] TS 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): tablet 2b4eec643b684e579575e67beff9a9e8 (table test-table [id=6d2541c63bba4747a538909e368010a1]) successfully deleted
I20260809 08:50:35.456766 3918 ts_tablet_manager.cc:1905] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:50:35.458256 3918 ts_tablet_manager.cc:1918] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:50:35.458331 3918 log.cc:1196] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/2b4eec643b684e579575e67beff9a9e8
I20260809 08:50:35.458575 3918 ts_tablet_manager.cc:1939] T 2b4eec643b684e579575e67beff9a9e8 P cc0658909f704494abff0400492e39c9: Deleting consensus metadata
I20260809 08:50:35.459012 3156 catalog_manager.cc:4841] TS cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): tablet 2b4eec643b684e579575e67beff9a9e8 (table test-table [id=6d2541c63bba4747a538909e368010a1]) successfully deleted
W20260809 08:50:35.461452 3159 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:40763 (error 108)
I20260809 08:50:35.461767 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 3472
I20260809 08:50:35.461897 3881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:35.461770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:40763 (local address 127.0.0.1:33948)
0809 08:50:35.461824 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:35.461864 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:40763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 08:50:35.475517 3159 catalog_manager.cc:4584] TS cf79186ef85a474abb1556f20cf23d62 (127.10.184.65:40763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b4eec643b684e579575e67beff9a9e8: Network error: Client connection negotiation failed: client connection to 127.10.184.65:40763: connect: Connection refused (error 111)
I20260809 08:50:35.479410 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 3682
I20260809 08:50:35.487242 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 3134
I20260809 08:50:55.501595 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--webserver_interface=127.10.184.126
--webserver_port=42127
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:55.581951 3920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:55.582144 3920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:55.582197 3920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:55.583304 3920 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 08:50:55.583341 3920 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 08:50:55.583357 3920 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 08:50:55.583372 3920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:50:55.583387 3920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:55.583405 3920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:55.583425 3920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:50:55.583447 3920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:50:55.584887 3920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=42127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:55.585150 3920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:55.585362 3920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:55.590188 3929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:55.590265 3932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:55.590327 3930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.590459 3920 server_base.cc:1034] running on GCE node
I20260809 08:50:55.590667 3920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:55.590930 3920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:55.592062 3920 hybrid_clock.cc:648] HybridClock initialized: now 1786265455592049 us; error 29 us; skew 500 ppm
Aug 09 08:50:55 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:55.594368 3920 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:50:55.594774 3920 webserver.cc:460] Webserver started at http://127.10.184.126:42127/ using document root <none> and password file <none>
I20260809 08:50:55.594978 3920 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:55.595026 3920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:55.596221 3920 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:55.596819 3940 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.596969 3920 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:55.597043 3920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9ea9b01e867448cf8647a344dd982099"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "c7ebb6cd8c69aa3bdcf6482aff67b61d"
server_key_iv: "4e757ccf1f008e63b2c147e0236fd35b"
server_key_version: "encryptionkey@0"
I20260809 08:50:55.597349 3920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:55.613468 3920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:55.615012 3920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:55.615131 3920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:55.618788 3920 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39307
I20260809 08:50:55.618820 4023 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39307 every 8 connection(s)
I20260809 08:50:55.619155 3920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:50:55.620487 4024 sys_catalog.cc:263] Verifying existing consensus state
I20260809 08:50:55.621165 4024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap starting.
I20260809 08:50:55.626806 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 3920
I20260809 08:50:55.627117 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:40763
--local_ip_for_outbound_sockets=127.10.184.65
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=35211
--webserver_interface=127.10.184.65
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:50:55.629850 4024 log.cc:824] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:55.631933 4024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:50:55.632187 4024 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Bootstrap complete.
I20260809 08:50:55.634207 4024 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:55.634415 4024 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Initialized, Role: FOLLOWER
I20260809 08:50:55.634552 4024 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:55.634606 4024 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:50:55.634634 4024 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:50:55.634696 4024 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 2 FOLLOWER]: Advancing to term 3
I20260809 08:50:55.635605 4024 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:55.635701 4024 leader_election.cc:302] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9ea9b01e867448cf8647a344dd982099; no voters:
I20260809 08:50:55.635852 4024 leader_election.cc:288] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 08:50:55.635900 4032 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 08:50:55.636025 4032 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [term 3 LEADER]: Becoming Leader. State: Replica: 9ea9b01e867448cf8647a344dd982099, State: Running, Role: LEADER
I20260809 08:50:55.636108 4024 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:50:55.636111 4032 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } }
I20260809 08:50:55.636451 4032 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9ea9b01e867448cf8647a344dd982099. Latest consensus state: current_term: 3 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:55.636531 4032 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:55.636480 4035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "9ea9b01e867448cf8647a344dd982099" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9ea9b01e867448cf8647a344dd982099" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39307 } } }
I20260809 08:50:55.636721 4041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:50:55.636716 4035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099 [sys.catalog]: This master's current role is: LEADER
I20260809 08:50:55.637682 4041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:50:55.637843 4041 catalog_manager.cc:1259] Loaded cluster ID: f10d41a1a30e40fda7a3988a3d3c81f4
I20260809 08:50:55.638005 4041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:50:55.639158 4041 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:50:55.639330 4041 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Loaded TSK: 0
I20260809 08:50:55.639379 4041 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9ea9b01e867448cf8647a344dd982099: Loaded TSK: 1
F20260809 08:50:55.639595 4041 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f76244439ad google::LogMessage::Fail() at ??:0
@ 0x7f762444688a google::LogMessage::SendToLog() at ??:0
@ 0x7f76244434de google::LogMessage::Flush() at ??:0
@ 0x7f7624444a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7626bb0104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f7626badd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f762c39d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f762c39d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f762c39d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f762c363c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f762c36dad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f76263110d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7626306954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f762716a6db start_thread at ??:0
@ 0x7f7622f4161f clone at ??:0
W20260809 08:50:55.726392 4029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:55.726576 4029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:55.726598 4029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:55.727715 4029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:55.727756 4029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:55.727779 4029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:50:55.729203 4029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:40763
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=35211
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:55.729463 4029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:55.729672 4029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:55.734910 4085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.735003 4029 server_base.cc:1034] running on GCE node
W20260809 08:50:55.735006 4088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:55.734916 4086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.735335 4029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:55.735561 4029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:55.736711 4029 hybrid_clock.cc:648] HybridClock initialized: now 1786265455736697 us; error 31 us; skew 500 ppm
Aug 09 08:50:55 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:55.739166 4029 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:50:55.739573 4029 webserver.cc:460] Webserver started at http://127.10.184.65:35211/ using document root <none> and password file <none>
I20260809 08:50:55.739769 4029 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:55.739817 4029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:55.741084 4029 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:55.741787 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.741956 4029 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:55.742062 4029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "cf79186ef85a474abb1556f20cf23d62"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "fe165eb42bfbe0c945634f60cfddc4e7"
server_key_iv: "ee90a1bbd35fc113f5c2496b3863af09"
server_key_version: "encryptionkey@0"
I20260809 08:50:55.742481 4029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:55.765470 4029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:55.767817 4029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:55.767967 4029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:55.768210 4029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:55.768736 4109 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 08:50:55.770218 4029 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 08:50:55.770278 4029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 08:50:55.770310 4029 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 08:50:55.771178 4029 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 08:50:55.771224 4029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 08:50:55.771283 4109 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Bootstrap starting.
I20260809 08:50:55.772920 4109 log.cc:824] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Log is configured to *not* fsync() on all Append() calls
I20260809 08:50:55.775790 4109 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 08:50:55.776073 4109 tablet_bootstrap.cc:492] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Bootstrap complete.
I20260809 08:50:55.776239 4109 ts_tablet_manager.cc:1397] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.005s
I20260809 08:50:55.778080 4029 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:40763
I20260809 08:50:55.777875 4109 raft_consensus.cc:348] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:55.778193 4109 raft_consensus.cc:729] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cf79186ef85a474abb1556f20cf23d62, State: Initialized, Role: FOLLOWER
I20260809 08:50:55.778294 4268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:40763 every 8 connection(s)
I20260809 08:50:55.778402 4109 consensus_queue.cc:260] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:55.778472 4029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:50:55.778674 4109 ts_tablet_manager.cc:1428] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:50:55.783046 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 4029
I20260809 08:50:55.783283 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:43039
--local_ip_for_outbound_sockets=127.10.184.66
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=45329
--webserver_interface=127.10.184.66
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:55.835685 4275 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:50:55.779482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:46279)
0809 08:50:55.779629 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.779660 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:55.779779 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:55.835505 (+ 55726us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
W20260809 08:50:55.835942 4273 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39307 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39307: Client connection negotiation failed: client connection to 127.10.184.126:39307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 08:50:55.836258 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:55.836152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:60603)
0809 08:50:55.836199 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.836238 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 08:50:55.836581 4275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:55.836503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:53119)
0809 08:50:55.836543 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.836565 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 08:50:55.836687 4273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 08:50:55.867798 4276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:55.868008 4276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:55.868042 4276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:55.869201 4276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:55.869246 4276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:55.869284 4276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:50:55.870723 4276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:43039
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=45329
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:55.870956 4276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:55.871168 4276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:55.876020 4291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:55.876011 4289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.876163 4276 server_base.cc:1034] running on GCE node
W20260809 08:50:55.876302 4287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.876484 4276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:55.876704 4276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:55.877833 4276 hybrid_clock.cc:648] HybridClock initialized: now 1786265455877820 us; error 30 us; skew 500 ppm
Aug 09 08:50:55 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:55.880103 4276 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:50:55.880473 4276 webserver.cc:460] Webserver started at http://127.10.184.66:45329/ using document root <none> and password file <none>
I20260809 08:50:55.880640 4276 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:55.880681 4276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:55.881907 4276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:55.882412 4303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.882562 4276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.882635 4276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "9e3c64bbe89f698314820fb33fbd1038"
server_key_iv: "60218ca3da604eb6ca6ace22c2693957"
server_key_version: "encryptionkey@0"
I20260809 08:50:55.882956 4276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:55.890576 4276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:55.892168 4276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:55.892287 4276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:55.892469 4276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:55.892747 4276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:55.892779 4276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.892805 4276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:55.892829 4276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.898243 4276 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43039
I20260809 08:50:55.898300 4483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43039 every 8 connection(s)
I20260809 08:50:55.898578 4276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:50:55.899501 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:55.899175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:39635)
0809 08:50:55.899304 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.899366 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260809 08:50:55.899835 4484 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39307 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39307: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
I20260809 08:50:55.900067 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:55.899992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:44329)
0809 08:50:55.900027 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.900051 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:50:55.900264 4486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:55.900205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:48721)
0809 08:50:55.900234 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:55.900251 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:50:55.900339 4484 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:50:55.901638 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 4276
I20260809 08:50:55.901849 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:36481
--local_ip_for_outbound_sockets=127.10.184.67
--tserver_master_addrs=127.10.184.126:39307
--webserver_port=40925
--webserver_interface=127.10.184.67
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:44941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:50:55.981863 4487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:50:55.982062 4487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:50:55.982090 4487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:50:55.983188 4487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:50:55.983222 4487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:50:55.983246 4487 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:50:55.984664 4487 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:44941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:36481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=40925
--tserver_master_addrs=127.10.184.126:39307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:50:55.984843 4487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:50:55.985072 4487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:50:55.989843 4495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.989919 4487 server_base.cc:1034] running on GCE node
W20260809 08:50:55.989822 4492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:50:55.989831 4493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:50:55.990199 4487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:50:55.990442 4487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:50:55.991612 4487 hybrid_clock.cc:648] HybridClock initialized: now 1786265455991597 us; error 24 us; skew 500 ppm
Aug 09 08:50:55 dist-test-slave-ncp9 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:50:55.993964 4487 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:50:55.994365 4487 webserver.cc:460] Webserver started at http://127.10.184.67:40925/ using document root <none> and password file <none>
I20260809 08:50:55.994561 4487 fs_manager.cc:359] Metadata directory not provided
I20260809 08:50:55.994609 4487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:50:55.995815 4487 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:50:55.996320 4507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:55.996450 4487 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:50:55.996533 4487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "cc0658909f704494abff0400492e39c9"
format_stamp: "Formatted at 2026-08-09 08:50:14 on dist-test-slave-ncp9"
server_key: "bf1f543914fbfedc5358c77fba97eb60"
server_key_iv: "0af70fbdad39d947e8c0cc854538fb89"
server_key_version: "encryptionkey@0"
I20260809 08:50:55.996837 4487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:50:56.013533 4487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:50:56.015206 4487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:50:56.015334 4487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:50:56.015519 4487 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:50:56.015801 4487 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:50:56.015841 4487 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:56.015867 4487 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:50:56.015889 4487 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:50:56.021413 4487 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:36481
I20260809 08:50:56.021473 4671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:36481 every 8 connection(s)
I20260809 08:50:56.021770 4487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:50:56.022682 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.022345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:54805)
0809 08:50:56.022487 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.022550 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":78,"threads_started":1}
W20260809 08:50:56.023041 4673 heartbeater.cc:644] Failed to heartbeat to 127.10.184.126:39307 (0 consecutive failures): Network error: Failed to ping master at 127.10.184.126:39307: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
I20260809 08:50:56.023281 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.023197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:34693)
0809 08:50:56.023241 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.023265 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:50:56.023469 4675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.023423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:51199)
0809 08:50:56.023442 (+ 19us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.023458 (+ 16us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10}
W20260809 08:50:56.023550 4673 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 08:50:56.026809 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 4487
I20260809 08:50:56.837216 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.837003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:52643)
0809 08:50:56.837116 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.837148 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":57,"threads_started":1}
I20260809 08:50:56.900774 4678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.900588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:37839)
0809 08:50:56.900694 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.900723 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260809 08:50:56.986189 4680 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:56.986315 4680 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:56.986681 4680 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
Aug 09 08:50:56 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.67@KRBTEST.COM
Aug 09 08:50:56 dist-test-slave-ncp9 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265455, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.66@KRBTEST.COM
I20260809 08:50:56.996842 4684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.986947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:35603 (local address 127.10.184.66:43039)
0809 08:50:56.987102 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:56.987115 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:56.987146 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:50:56.987724 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:56.987727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:56.987766 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:56.987920 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:56.988795 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.989774 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.990802 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.991200 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.991294 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:56.993889 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:56.993893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:56.993893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:56.993905 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:56.995391 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995726 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:56.995729 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:56.995729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:56.995812 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:56.996023 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:56.996024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:56.996024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:56.996256 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:56.996330 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:56.996610 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 08:50:56.996733 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":52,"threads_started":1}
I20260809 08:50:56.996869 4682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.986823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43039 (local address 127.10.184.65:35603)
0809 08:50:56.987000 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.987008 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:56.987038 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:56.988050 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:56.988051 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:56.988421 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:56.988647 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:56.988654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.989873 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.989874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:56.990716 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:56.990719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.991284 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.991285 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:56.991394 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:56.991946 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:56.991951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:56.993794 (+ 1843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:56.995464 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:56.995465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:56.995646 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:56.995875 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:56.995876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:56.995963 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:56.996367 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:56.996368 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:56.996458 (+ 90us) client_negotiation.cc:769] Sending connection context
0809 08:50:56.996546 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:50:56.996736 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":73,"thread_start_us":86,"threads_started":1}
I20260809 08:50:56.996974 4677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.986822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:36481 (local address 127.10.184.65:43793)
0809 08:50:56.986860 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:56.986870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:50:56.986924 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:50:56.987999 (+ 1075us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:50:56.988003 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:50:56.988423 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:50:56.988861 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:56.988864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.989990 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.989991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:56.990715 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:50:56.990717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.991276 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.991277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:50:56.991393 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:56.991903 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:50:56.991911 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:50:56.993413 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:50:56.995279 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995286 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:56.995289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:56.995579 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:56.995850 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995851 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:50:56.995851 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:50:56.995963 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:50:56.996325 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:50:56.996332 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:50:56.996457 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 08:50:56.996545 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:50:56.996812 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":133}
I20260809 08:50:56.996852 4683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:56.986942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:43793 (local address 127.10.184.67:36481)
0809 08:50:56.987102 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 08:50:56.987115 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:50:56.987141 (+ 26us) server_negotiation.cc:407] Connection header received
0809 08:50:56.987671 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:50:56.987674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:50:56.987760 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:50:56.987927 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:50:56.989002 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.989894 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.990797 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:50:56.991195 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:50:56.991290 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:50:56.993518 (+ 2228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:50:56.993523 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:50:56.993523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:50:56.993532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:50:56.995199 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:56.995670 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:56.995674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:56.995674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:56.995787 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:50:56.996025 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:50:56.996026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:50:56.996027 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:50:56.996257 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:50:56.996330 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 08:50:56.996615 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 08:50:56.996746 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":55,"threads_started":1}
W20260809 08:50:56.997709 4102 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
W20260809 08:50:56.998095 4101 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
I20260809 08:50:56.998164 4101 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf79186ef85a474abb1556f20cf23d62; no voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9
I20260809 08:50:56.998287 4680 raft_consensus.cc:2726] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:57.024029 4685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:57.023810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:55765)
0809 08:50:57.023919 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:57.023987 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":47,"threads_started":1}
I20260809 08:50:57.837787 4688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:57.837591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:59223)
0809 08:50:57.837707 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:57.837742 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":60,"threads_started":1}
I20260809 08:50:57.901355 4689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:57.901170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:36831)
0809 08:50:57.901283 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:57.901318 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":61,"threads_started":1}
I20260809 08:50:58.025383 4690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:58.025184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:57867)
0809 08:50:58.025298 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:58.025330 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":63,"threads_started":1}
I20260809 08:50:58.838389 4692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:58.838195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:49743)
0809 08:50:58.838311 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:58.838344 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":65,"threads_started":1}
I20260809 08:50:58.901891 4693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:58.901713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:58819)
0809 08:50:58.901819 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:58.901854 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260809 08:50:59.025938 4695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:59.025752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:44209)
0809 08:50:59.025863 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:59.025898 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":61,"threads_started":1}
I20260809 08:50:59.122393 4696 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:50:59.122490 4696 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:50:59.122659 4696 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
W20260809 08:50:59.123025 4102 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
W20260809 08:50:59.123154 4101 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
I20260809 08:50:59.123224 4101 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf79186ef85a474abb1556f20cf23d62; no voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9
I20260809 08:50:59.123330 4696 raft_consensus.cc:2726] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:50:59.838973 4698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:59.838788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:53769)
0809 08:50:59.838897 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:59.838928 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":60,"threads_started":1}
I20260809 08:50:59.902438 4699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:50:59.902245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:40833)
0809 08:50:59.902366 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:50:59.902401 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":73,"threads_started":1}
I20260809 08:51:00.026494 4700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:00.026295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:57595)
0809 08:51:00.026422 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:00.026456 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260809 08:51:00.839564 4701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:00.839358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:36739)
0809 08:51:00.839487 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:00.839519 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260809 08:51:00.902988 4702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:00.902795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:46337)
0809 08:51:00.902916 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:00.902950 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":74,"threads_started":1}
I20260809 08:51:01.027046 4703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:01.026853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:54323)
0809 08:51:01.026977 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:01.027007 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 08:51:01.840330 4704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:01.840098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:34541)
0809 08:51:01.840248 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:01.840284 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260809 08:51:01.857538 4709 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:01.857633 4709 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:51:01.857803 4709 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
W20260809 08:51:01.858175 4101 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
W20260809 08:51:01.858250 4102 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
I20260809 08:51:01.858305 4102 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf79186ef85a474abb1556f20cf23d62; no voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9
I20260809 08:51:01.858382 4709 raft_consensus.cc:2726] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:51:01.903499 4710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:01.903319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:41077)
0809 08:51:01.903428 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:01.903462 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260809 08:51:02.027607 4712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:02.027414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:44641)
0809 08:51:02.027538 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:02.027568 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260809 08:51:02.841025 4713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:02.840762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:55023)
0809 08:51:02.840928 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:02.840978 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":59,"threads_started":1}
I20260809 08:51:02.904052 4714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:02.903857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:51287)
0809 08:51:02.903976 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:02.904011 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":60,"threads_started":1}
I20260809 08:51:03.028256 4715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:03.028052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:56713)
0809 08:51:03.028184 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:03.028216 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":66,"threads_started":1}
I20260809 08:51:03.841634 4718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:03.841416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:55717)
0809 08:51:03.841546 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:03.841586 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":64,"threads_started":1}
I20260809 08:51:03.904639 4719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:03.904437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:34389)
0809 08:51:03.904560 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:03.904599 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":75,"threads_started":1}
I20260809 08:51:04.028846 4720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:04.028648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:35835)
0809 08:51:04.028773 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:04.028807 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260809 08:51:04.446827 4721 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:04.446925 4721 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:51:04.447114 4721 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
W20260809 08:51:04.447454 4101 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
W20260809 08:51:04.447538 4102 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
I20260809 08:51:04.447579 4102 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf79186ef85a474abb1556f20cf23d62; no voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9
I20260809 08:51:04.447650 4721 raft_consensus.cc:2726] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:51:04.842279 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:04.842058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:51501)
0809 08:51:04.842201 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:04.842234 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":79,"threads_started":1}
I20260809 08:51:04.905200 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:04.904991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:42683)
0809 08:51:04.905108 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:04.905140 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260809 08:51:05.029408 4725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:05.029220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:51643)
0809 08:51:05.029339 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:05.029370 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
I20260809 08:51:05.842912 4726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:05.842721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:48741)
0809 08:51:05.842835 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:05.842868 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":63,"threads_started":1}
I20260809 08:51:05.905778 4727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:05.905567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:40859)
0809 08:51:05.905700 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:05.905736 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260809 08:51:06.029997 4728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:06.029809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:52331)
0809 08:51:06.029923 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:06.029957 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260809 08:51:06.843511 4729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:06.843316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:42079)
0809 08:51:06.843430 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:06.843464 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":59,"threads_started":1}
I20260809 08:51:06.906315 4731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:06.906124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:55963)
0809 08:51:06.906248 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:06.906277 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260809 08:51:07.030577 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:07.030362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:40165)
0809 08:51:07.030493 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:07.030531 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260809 08:51:07.844040 4733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:07.843856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:56757)
0809 08:51:07.843968 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:07.844001 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260809 08:51:07.906870 4737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:07.906673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:36587)
0809 08:51:07.906784 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:07.906815 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260809 08:51:08.031174 4738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:08.030988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:50897)
0809 08:51:08.031100 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:08.031133 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":59,"threads_started":1}
I20260809 08:51:08.581077 4739 raft_consensus.cc:482] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:08.581190 4739 raft_consensus.cc:504] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc0658909f704494abff0400492e39c9" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 36481 } } peers { permanent_uuid: "8c9a96c37e0a49cdb987a1eeeccf1f15" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43039 } } peers { permanent_uuid: "cf79186ef85a474abb1556f20cf23d62" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 40763 } }
I20260809 08:51:08.581329 4739 leader_election.cc:288] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cc0658909f704494abff0400492e39c9 (127.10.184.67:36481), 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039)
W20260809 08:51:08.581687 4101 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer cc0658909f704494abff0400492e39c9 (127.10.184.67:36481): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
W20260809 08:51:08.581771 4102 leader_election.cc:341] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c9a96c37e0a49cdb987a1eeeccf1f15 (127.10.184.66:43039): Not found: Tablet not found: 2b4eec643b684e579575e67beff9a9e8
I20260809 08:51:08.581810 4102 leader_election.cc:302] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cf79186ef85a474abb1556f20cf23d62; no voters: 8c9a96c37e0a49cdb987a1eeeccf1f15, cc0658909f704494abff0400492e39c9
I20260809 08:51:08.581877 4739 raft_consensus.cc:2726] T 2b4eec643b684e579575e67beff9a9e8 P cf79186ef85a474abb1556f20cf23d62 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 08:51:08.844627 4740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:08.844428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:53717)
0809 08:51:08.844548 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:08.844583 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260809 08:51:08.907428 4741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:08.907247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:54237)
0809 08:51:08.907357 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:08.907388 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":59,"threads_started":1}
I20260809 08:51:09.031734 4742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:09.031537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:50051)
0809 08:51:09.031656 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:09.031694 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260809 08:51:09.845254 4743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:09.845038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:53191)
0809 08:51:09.845160 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:09.845205 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260809 08:51:09.907995 4747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:09.907802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:60437)
0809 08:51:09.907918 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:09.907952 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260809 08:51:10.032302 4748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:10.032092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:41645)
0809 08:51:10.032221 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:10.032258 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":65,"threads_started":1}
I20260809 08:51:10.845901 4751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:10.845697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.65:46167)
0809 08:51:10.845823 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:10.845857 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260809 08:51:10.908636 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:10.908392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.66:60137)
0809 08:51:10.908545 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:10.908587 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":73,"threads_started":1}
I20260809 08:51:11.028493 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 4029
I20260809 08:51:11.032953 4755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:11.032754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39307 (local address 127.10.184.67:45605)
0809 08:51:11.032862 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:11.032907 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.126:39307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":54,"threads_started":1}
I20260809 08:51:11.035900 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 4276
I20260809 08:51:11.044252 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 4487
2026-08-09T08:51:11Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58311 ms)
[----------] 30 tests from SecurityITest (229659 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:11 dist-test-slave-ncp9 krb5kdc[4760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:11 dist-test-slave-ncp9 krb5kdc[4760](info): set up 2 sockets
Aug 09 08:51:11 dist-test-slave-ncp9 krb5kdc[4760](info): commencing operation
krb5kdc: starting...
W20260809 08:51:12.187160 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:12.297214 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:43495
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:43495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:12.376060 4775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:12.376266 4775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:12.376287 4775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:12.377408 4775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:12.377444 4775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:12.377456 4775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:12.377467 4775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:12.377485 4775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:12.378877 4775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:43495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:43495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4775
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:12.379062 4775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:12.379235 4775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:12.383970 4775 server_base.cc:1034] running on GCE node
W20260809 08:51:12.383951 4784 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.383891 4788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.383891 4785 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:12.384500 4775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:12.384742 4775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:12.385890 4775 hybrid_clock.cc:648] HybridClock initialized: now 1786265472385877 us; error 27 us; skew 500 ppm
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:12.388304 4775 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:12.388722 4775 webserver.cc:460] Webserver started at http://127.10.184.126:36455/ using document root <none> and password file <none>
I20260809 08:51:12.388913 4775 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:12.388967 4775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:12.389060 4775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:12.390018 4775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "79ede5ae2093431ebec1e730f685dc60"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "b2bdb678c3134843876fc67d6b4658b2"
server_key_iv: "a1cf203fa4a3297fbe38f9050021efa9"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.390316 4775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "79ede5ae2093431ebec1e730f685dc60"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "b2bdb678c3134843876fc67d6b4658b2"
server_key_iv: "a1cf203fa4a3297fbe38f9050021efa9"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.391496 4775 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:12.392086 4798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.392256 4775 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:12.392346 4775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "79ede5ae2093431ebec1e730f685dc60"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "b2bdb678c3134843876fc67d6b4658b2"
server_key_iv: "a1cf203fa4a3297fbe38f9050021efa9"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.392413 4775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:12.408674 4775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:12.410458 4775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:12.410573 4775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:12.414068 4775 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:43495
I20260809 08:51:12.414124 4865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:43495 every 8 connection(s)
I20260809 08:51:12.414433 4775 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:12.415262 4866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:12.417656 4866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: Bootstrap starting.
I20260809 08:51:12.418248 4866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:12.418480 4866 log.cc:824] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:12.419111 4866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: No bootstrap required, opened a new log
I20260809 08:51:12.420652 4866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } }
I20260809 08:51:12.420783 4866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:12.420811 4866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ede5ae2093431ebec1e730f685dc60, State: Initialized, Role: FOLLOWER
I20260809 08:51:12.420945 4866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } }
I20260809 08:51:12.421023 4866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:12.421061 4866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:12.421113 4866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:12.421196 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 4775
I20260809 08:51:12.421305 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:12.421504 10977 external_mini_cluster.cc:1428] Setting key 98979c52e9396269ad45ec57416c7298
I20260809 08:51:12.421751 4866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } }
I20260809 08:51:12.421854 4866 leader_election.cc:302] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ede5ae2093431ebec1e730f685dc60; no voters:
I20260809 08:51:12.422020 4866 leader_election.cc:288] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:12.422060 4874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:12.422241 4874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [term 1 LEADER]: Becoming Leader. State: Replica: 79ede5ae2093431ebec1e730f685dc60, State: Running, Role: LEADER
I20260809 08:51:12.422362 4866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:12.422367 4874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } }
I20260809 08:51:12.422945 4876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ede5ae2093431ebec1e730f685dc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } } }
I20260809 08:51:12.423391 4876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:12.423281 4877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ede5ae2093431ebec1e730f685dc60. Latest consensus state: current_term: 1 leader_uuid: "79ede5ae2093431ebec1e730f685dc60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ede5ae2093431ebec1e730f685dc60" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 43495 } } }
I20260809 08:51:12.423573 4877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60 [sys.catalog]: This master's current role is: LEADER
W20260809 08:51:12.424134 4897 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:12.424189 4897 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:12.424240 4895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:12.424379 4895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:12.425936 4895 catalog_manager.cc:1347] Generated new cluster ID: c648e36ef08043a8ace9259f603c5750
I20260809 08:51:12.426002 4895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:12.432577 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.421913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.10.184.126:43495)
0809 08:51:12.422085 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.422094 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.422281 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:51:12.422419 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.422426 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.422480 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.422617 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:12.423391 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.424657 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.425656 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.426181 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.426282 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.428420 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:12.428424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:12.428424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:12.428436 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:12.430757 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.431140 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.431154 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.431154 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.431287 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.431504 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.431505 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.431505 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.431811 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:12.431945 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.432220 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.432367 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260809 08:51:12.433516 4895 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:12.434304 4895 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:12.445839 4895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: Generated new TSK 0
I20260809 08:51:12.445997 4895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:51:12.469686 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:12.548521 4903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:12.548714 4903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:12.548744 4903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:12.549893 4903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:12.549937 4903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:12.549971 4903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:12.551383 4903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:12.551599 4903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:12.551795 4903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:12.556586 4914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.556597 4917 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.556635 4910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:12.556632 4903 server_base.cc:1034] running on GCE node
I20260809 08:51:12.557003 4903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:12.557256 4903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:12.558384 4903 hybrid_clock.cc:648] HybridClock initialized: now 1786265472558375 us; error 27 us; skew 500 ppm
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:12.560719 4903 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:51:12.561136 4903 webserver.cc:460] Webserver started at http://127.10.184.65:33713/ using document root <none> and password file <none>
I20260809 08:51:12.561340 4903 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:12.561379 4903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:12.561468 4903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:12.562428 4903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "7a29060e6912440bb8ea849c263f62ce"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "7563668bcd559c0e99301e837d96a3a8"
server_key_iv: "4cc71ff10261c513397f627794d83e83"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.562721 4903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "7a29060e6912440bb8ea849c263f62ce"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "7563668bcd559c0e99301e837d96a3a8"
server_key_iv: "4cc71ff10261c513397f627794d83e83"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.563866 4903 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:51:12.564479 4928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.564625 4903 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.564694 4903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "7a29060e6912440bb8ea849c263f62ce"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "7563668bcd559c0e99301e837d96a3a8"
server_key_iv: "4cc71ff10261c513397f627794d83e83"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.564759 4903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:12.578088 4903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:12.580045 4903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:12.580176 4903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:12.580384 4903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:12.580672 4903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:12.580705 4903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.580725 4903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:12.580747 4903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.586221 4903 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:42591
I20260809 08:51:12.586257 5092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:42591 every 8 connection(s)
I20260809 08:51:12.586531 4903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:12.594058 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 4903
I20260809 08:51:12.594161 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:12.594331 10977 external_mini_cluster.cc:1428] Setting key 5f494ca1e77fb624b31a34a957bc8982
I20260809 08:51:12.594833 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.587157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:59291 (local address 127.10.184.126:43495)
0809 08:51:12.587205 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.587207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.587389 (+ 182us) server_negotiation.cc:407] Connection header received
0809 08:51:12.587849 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.587850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.587860 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.587904 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:12.588744 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.589298 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.590210 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.590398 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.590441 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.592265 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:12.592266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:12.592266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:12.592273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:12.593498 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.593863 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.593864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.593864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.593917 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.594130 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.594131 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.594132 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.594271 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:12.594331 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.594669 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.594743 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:51:12.594997 5102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.587110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43495 (local address 127.10.184.65:59291)
0809 08:51:12.587291 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.587325 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.587773 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.587961 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.587964 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.588491 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:12.588677 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.588682 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.589363 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.589364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.590144 (+ 780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.590146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.590450 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.590451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.590543 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.590912 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:12.590919 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:12.592173 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:12.593573 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.593580 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.593582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.593794 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.593972 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.593972 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.593973 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.594075 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.594332 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:12.594340 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:12.594484 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.594572 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.594742 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":121,"threads_started":1}
I20260809 08:51:12.595487 5095 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43495
I20260809 08:51:12.595583 5095 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:12.595759 5095 heartbeater.cc:511] Master 127.10.184.126:43495 requested a full tablet report, sending...
I20260809 08:51:12.596256 4820 ts_manager.cc:194] Registered new tserver with Master: 7a29060e6912440bb8ea849c263f62ce (127.10.184.65:42591)
I20260809 08:51:12.596827 4820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:59291
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:51:12.617774 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:12.697072 5106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:12.697252 5106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:12.697273 5106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:12.698346 5106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:12.698382 5106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:12.698407 5106 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:12.699779 5106 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5106
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:12.699949 5106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:12.700114 5106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:12.704908 5115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.704939 5114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.704939 5117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:12.704972 5106 server_base.cc:1034] running on GCE node
I20260809 08:51:12.705286 5106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:12.705495 5106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:12.706619 5106 hybrid_clock.cc:648] HybridClock initialized: now 1786265472706607 us; error 30 us; skew 500 ppm
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:12.708851 5106 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:51:12.709280 5106 webserver.cc:460] Webserver started at http://127.10.184.66:37177/ using document root <none> and password file <none>
I20260809 08:51:12.709470 5106 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:12.709508 5106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:12.709597 5106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:12.710532 5106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "75c07559595c443082801cf92e422169"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "653457660885bfd409edb213b1ed17c3"
server_key_iv: "c0175969b8130c60772010d680c70303"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.710834 5106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "75c07559595c443082801cf92e422169"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "653457660885bfd409edb213b1ed17c3"
server_key_iv: "c0175969b8130c60772010d680c70303"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.711948 5106 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 08:51:12.712549 5128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.712697 5106 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.712764 5106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "75c07559595c443082801cf92e422169"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "653457660885bfd409edb213b1ed17c3"
server_key_iv: "c0175969b8130c60772010d680c70303"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.712826 5106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:12.730719 5106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:12.732270 5106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:12.732378 5106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:12.732540 5106 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:12.732775 5106 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:12.732806 5106 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.732825 5106 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:12.732842 5106 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.738044 5106 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43195
I20260809 08:51:12.738106 5291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43195 every 8 connection(s)
I20260809 08:51:12.738374 5106 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:12.742069 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 5106
I20260809 08:51:12.742198 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:12.742391 10977 external_mini_cluster.cc:1428] Setting key 4f1e7d4c22af95fe23c798399bc73de9
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:12.747229 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.739030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:35677 (local address 127.10.184.126:43495)
0809 08:51:12.739080 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.739082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.739261 (+ 179us) server_negotiation.cc:407] Connection header received
0809 08:51:12.739695 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.739696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.739706 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.739750 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:12.740576 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.741117 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.742120 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.742361 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.742413 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.744607 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:12.744608 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:12.744609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:12.744615 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:12.746003 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.746417 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.746418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.746418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.746468 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.746697 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.746698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.746698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.746789 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:12.746834 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.747106 (+ 272us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.747166 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:51:12.747483 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.738997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43495 (local address 127.10.184.66:35677)
0809 08:51:12.739154 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.739188 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.739612 (+ 424us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.739806 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.739809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.740337 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:12.740513 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.740518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.741182 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.741183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.742051 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.742054 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.742438 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.742439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.742583 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.743027 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:12.743037 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:12.744520 (+ 1483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:12.746075 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.746082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.746084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.746352 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.746518 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.746519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.746519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.746645 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.746838 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:12.746843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:12.746991 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.747096 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.747261 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260809 08:51:12.747965 5292 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43495
I20260809 08:51:12.748061 5292 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:12.748256 5292 heartbeater.cc:511] Master 127.10.184.126:43495 requested a full tablet report, sending...
I20260809 08:51:12.748661 4820 ts_manager.cc:194] Registered new tserver with Master: 75c07559595c443082801cf92e422169 (127.10.184.66:43195)
I20260809 08:51:12.749074 4820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:35677
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:51:12.767686 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:12.846267 5299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:12.846449 5299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:12.846470 5299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:12.847522 5299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:12.847553 5299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:12.847577 5299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:12.848950 5299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:43495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:12.849113 5299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:12.849277 5299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:12.853957 5305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:12.854074 5299 server_base.cc:1034] running on GCE node
W20260809 08:51:12.853967 5310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:12.853967 5307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:12.854368 5299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:12.854589 5299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:12.855741 5299 hybrid_clock.cc:648] HybridClock initialized: now 1786265472855733 us; error 34 us; skew 500 ppm
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:12.858075 5299 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:51:12.858480 5299 webserver.cc:460] Webserver started at http://127.10.184.67:34263/ using document root <none> and password file <none>
I20260809 08:51:12.858669 5299 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:12.858714 5299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:12.858836 5299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:12.859766 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "2d45599b92ba44e1aefd173322b03078"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "341a6e8721d47ab8e6a9049caa138b86"
server_key_iv: "5220020cb4f49103601a94b478c55187"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.860152 5299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "2d45599b92ba44e1aefd173322b03078"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "341a6e8721d47ab8e6a9049caa138b86"
server_key_iv: "5220020cb4f49103601a94b478c55187"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.861406 5299 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260809 08:51:12.862052 5321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.862201 5299 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.862298 5299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "2d45599b92ba44e1aefd173322b03078"
format_stamp: "Formatted at 2026-08-09 08:51:12 on dist-test-slave-ncp9"
server_key: "341a6e8721d47ab8e6a9049caa138b86"
server_key_iv: "5220020cb4f49103601a94b478c55187"
server_key_version: "encryptionkey@0"
I20260809 08:51:12.862355 5299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:12.884516 5299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:12.886449 5299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:12.886602 5299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:12.886791 5299 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:12.887068 5299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:12.887102 5299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.887128 5299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:12.887145 5299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:12.892486 5299 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:35099
I20260809 08:51:12.892541 5504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:35099 every 8 connection(s)
I20260809 08:51:12.892797 5299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:51:12 dist-test-slave-ncp9 krb5kdc[4760](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265472, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:12.901170 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.893485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:40299 (local address 127.10.184.126:43495)
0809 08:51:12.893538 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.893540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.893709 (+ 169us) server_negotiation.cc:407] Connection header received
0809 08:51:12.894196 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.894198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.894211 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.894257 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:12.895227 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.895789 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.896750 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.896997 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.897048 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.898911 (+ 1863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:12.898912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:12.898913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:12.898920 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:12.899965 (+ 1045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.900316 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.900317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.900318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.900375 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.900637 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.900638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.900638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.900748 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:12.900799 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.901044 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.901107 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:51:12.901337 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.893458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:43495 (local address 127.10.184.67:40299)
0809 08:51:12.893598 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.893636 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.894103 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.894338 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.894342 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.894944 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:12.895133 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.895138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.895871 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.895872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.896666 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.896669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.897073 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.897074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.897192 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.897545 (+ 353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:12.897553 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:12.898820 (+ 1267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:12.900031 (+ 1211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.900036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.900038 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.900240 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.900452 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:12.900453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:12.900462 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:12.900574 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:12.900796 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:12.900798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:12.900930 (+ 132us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.901026 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.901144 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260809 08:51:12.901744 5505 heartbeater.cc:347] Connected to a master server at 127.10.184.126:43495
I20260809 08:51:12.901824 5505 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:12.901978 5505 heartbeater.cc:511] Master 127.10.184.126:43495 requested a full tablet report, sending...
I20260809 08:51:12.902268 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 5299
I20260809 08:51:12.902333 4820 ts_manager.cc:194] Registered new tserver with Master: 2d45599b92ba44e1aefd173322b03078 (127.10.184.67:35099)
I20260809 08:51:12.902379 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:12.902545 10977 external_mini_cluster.cc:1428] Setting key 1e3044ad0bfe5092cc832eb68039a1ac
I20260809 08:51:12.902737 4820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:40299
I20260809 08:51:12.903374 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:12.909813 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.904605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.10.184.126:43495)
0809 08:51:12.904642 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.904644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.904747 (+ 103us) server_negotiation.cc:407] Connection header received
0809 08:51:12.904804 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.904805 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.904814 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.904858 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:12.905416 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.905862 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.906562 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.906751 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.906801 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.907837 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:12.907838 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:12.907839 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:12.907846 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:12.908837 (+ 991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.909142 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.909143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.909143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.909198 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:12.909355 (+ 157us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:12.909356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:12.909356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:12.909492 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:12.909583 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.909700 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.909757 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:12.911188 4819 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:12.912729 4819 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:12.918704 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:42591 (local address 127.0.0.1:37046)
0809 08:51:12.915076 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.915085 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.915135 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.915424 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.915425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.915427 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:12.915468 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.915469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916691 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.916692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.917599 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.917602 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.918430 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.918431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.918507 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.918520 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.918587 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.918635 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":60,"threads_started":1}
I20260809 08:51:12.918733 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37046 (local address 127.10.184.65:42591)
0809 08:51:12.915103 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.915108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.915127 (+ 19us) server_negotiation.cc:407] Connection header received
0809 08:51:12.915231 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.915233 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.915250 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.915319 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:12.915542 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916609 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.917670 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.918344 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.918421 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.918476 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.918613 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.918654 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":92,"threads_started":1}
I20260809 08:51:12.918805 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41270 (local address 127.10.184.67:35099)
0809 08:51:12.914798 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.914802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.915089 (+ 287us) server_negotiation.cc:407] Connection header received
0809 08:51:12.915189 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.915191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.915206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.915277 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:12.915426 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916546 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.917859 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.918522 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.918575 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.918623 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.918705 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.918739 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":77,"threads_started":1}
I20260809 08:51:12.918787 5526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:35099 (local address 127.0.0.1:41270)
0809 08:51:12.915030 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.915040 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.915128 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.915285 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.915288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.915290 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:12.915351 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.915352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916646 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.916646 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.917790 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.917792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.918589 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.918590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.918645 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.918649 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.918697 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.918729 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"mutex_wait_us":24,"thread_start_us":60,"threads_started":1}
I20260809 08:51:12.919302 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.10.184.66:43195)
0809 08:51:12.914955 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:12.914968 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:12.914980 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:51:12.915258 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:12.915261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:12.915276 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:12.915343 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:12.915563 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916417 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.917887 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.918355 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.918423 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.918489 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:12.919191 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 08:51:12.919231 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260809 08:51:12.919260 5518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:12.914544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43195 (local address 127.0.0.1:49552)
0809 08:51:12.914893 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:12.914930 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:12.914980 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:12.915425 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:12.915426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:12.915428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:12.915482 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.915484 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.916513 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.916513 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.917783 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:12.917787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:12.919066 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:12.919067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:12.919111 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:12.919114 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:12.919160 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 08:51:12.919199 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":1,"thread_start_us":63,"threads_started":1}
I20260809 08:51:12.919997 4989 tablet_service.cc:1467] Processing CreateTablet for tablet cb9e5198857a41278343e686222dac12 (DEFAULT_TABLE table=test-table [id=419659ceb6504b0dafa203cd4f293271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:12.920039 5196 tablet_service.cc:1467] Processing CreateTablet for tablet cb9e5198857a41278343e686222dac12 (DEFAULT_TABLE table=test-table [id=419659ceb6504b0dafa203cd4f293271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:12.920280 4989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9e5198857a41278343e686222dac12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:12.920291 5196 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9e5198857a41278343e686222dac12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:12.921247 5400 tablet_service.cc:1467] Processing CreateTablet for tablet cb9e5198857a41278343e686222dac12 (DEFAULT_TABLE table=test-table [id=419659ceb6504b0dafa203cd4f293271]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:12.921514 5400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cb9e5198857a41278343e686222dac12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:12.923065 5529 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Bootstrap starting.
I20260809 08:51:12.923792 5529 tablet_bootstrap.cc:654] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:12.924425 5529 log.cc:824] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:12.924422 5530 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Bootstrap starting.
I20260809 08:51:12.925117 5530 tablet_bootstrap.cc:654] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:12.925272 5529 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: No bootstrap required, opened a new log
I20260809 08:51:12.925349 5529 ts_tablet_manager.cc:1397] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:12.925594 5530 log.cc:824] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:12.926314 5530 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: No bootstrap required, opened a new log
I20260809 08:51:12.926373 5530 ts_tablet_manager.cc:1397] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:12.926918 5534 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: Bootstrap starting.
I20260809 08:51:12.927209 5529 raft_consensus.cc:348] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.927345 5529 raft_consensus.cc:374] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:12.927376 5529 raft_consensus.cc:729] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75c07559595c443082801cf92e422169, State: Initialized, Role: FOLLOWER
I20260809 08:51:12.927506 5529 consensus_queue.cc:260] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.927592 5534 tablet_bootstrap.cc:654] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:12.927773 5529 ts_tablet_manager.cc:1428] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:51:12.927794 5292 heartbeater.cc:503] Master 127.10.184.126:43495 was elected leader, sending a full tablet report...
I20260809 08:51:12.928153 5534 log.cc:824] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:12.928126 5530 raft_consensus.cc:348] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.928285 5530 raft_consensus.cc:374] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:12.928331 5530 raft_consensus.cc:729] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d45599b92ba44e1aefd173322b03078, State: Initialized, Role: FOLLOWER
I20260809 08:51:12.928478 5530 consensus_queue.cc:260] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.928726 5505 heartbeater.cc:503] Master 127.10.184.126:43495 was elected leader, sending a full tablet report...
I20260809 08:51:12.928733 5530 ts_tablet_manager.cc:1428] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:12.929097 5534 tablet_bootstrap.cc:492] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: No bootstrap required, opened a new log
I20260809 08:51:12.929167 5534 ts_tablet_manager.cc:1397] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:12.930552 5534 raft_consensus.cc:348] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.930675 5534 raft_consensus.cc:374] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:12.930696 5534 raft_consensus.cc:729] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a29060e6912440bb8ea849c263f62ce, State: Initialized, Role: FOLLOWER
I20260809 08:51:12.930797 5534 consensus_queue.cc:260] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:12.931030 5534 ts_tablet_manager.cc:1428] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:51:12.931066 5095 heartbeater.cc:503] Master 127.10.184.126:43495 was elected leader, sending a full tablet report...
W20260809 08:51:12.988922 5293 tablet.cc:2369] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:51:12.999990 5542 raft_consensus.cc:482] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:13.000111 5542 raft_consensus.cc:504] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:13.000449 5542 leader_election.cc:288] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75c07559595c443082801cf92e422169 (127.10.184.66:43195), 7a29060e6912440bb8ea849c263f62ce (127.10.184.65:42591)
I20260809 08:51:13.003100 5524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.000666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:51791 (local address 127.10.184.66:43195)
0809 08:51:13.000710 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.000712 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.000723 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:51:13.000758 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.000759 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.000771 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.000829 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.001017 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.001485 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002418 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.002794 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002856 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.002865 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.003005 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.003038 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:13.003100 5509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.000607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43195 (local address 127.10.184.67:51791)
0809 08:51:13.000654 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.000667 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.000710 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.000837 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.000838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.000841 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.000938 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.000940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.001557 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.001558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.002324 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.002326 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.002864 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.002939 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.002942 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.002986 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.003027 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15,"mutex_wait_us":25}
I20260809 08:51:13.003207 5547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.000609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:42591 (local address 127.10.184.67:33993)
0809 08:51:13.000792 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.000799 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.000844 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.000976 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.000977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.000978 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.001044 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.001045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.001658 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.001659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.002412 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.002414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.002939 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002941 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.003003 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.003007 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.003050 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.003089 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":50,"thread_start_us":87,"threads_started":1}
I20260809 08:51:13.003194 5528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.000705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:33993 (local address 127.10.184.65:42591)
0809 08:51:13.000745 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.000747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.000857 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:51:13.000910 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.000911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.000924 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.000969 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.001096 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.001582 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002492 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.002868 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.002929 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.002937 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.003061 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.003122 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:13.003568 5224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9e5198857a41278343e686222dac12" candidate_uuid: "2d45599b92ba44e1aefd173322b03078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c07559595c443082801cf92e422169" is_pre_election: true
I20260809 08:51:13.003593 5019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9e5198857a41278343e686222dac12" candidate_uuid: "2d45599b92ba44e1aefd173322b03078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a29060e6912440bb8ea849c263f62ce" is_pre_election: true
I20260809 08:51:13.003697 5224 raft_consensus.cc:2454] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d45599b92ba44e1aefd173322b03078 in term 0.
I20260809 08:51:13.003726 5019 raft_consensus.cc:2454] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2d45599b92ba44e1aefd173322b03078 in term 0.
I20260809 08:51:13.003927 5323 leader_election.cc:302] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d45599b92ba44e1aefd173322b03078, 75c07559595c443082801cf92e422169; no voters:
I20260809 08:51:13.004092 5542 raft_consensus.cc:2781] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:13.004137 5542 raft_consensus.cc:482] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:13.004155 5542 raft_consensus.cc:3037] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.005040 5542 raft_consensus.cc:504] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:13.005152 5542 leader_election.cc:288] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [CANDIDATE]: Term 1 election: Requested vote from peers 75c07559595c443082801cf92e422169 (127.10.184.66:43195), 7a29060e6912440bb8ea849c263f62ce (127.10.184.65:42591)
I20260809 08:51:13.005296 5019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9e5198857a41278343e686222dac12" candidate_uuid: "2d45599b92ba44e1aefd173322b03078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a29060e6912440bb8ea849c263f62ce"
I20260809 08:51:13.005364 5019 raft_consensus.cc:3037] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.006023 5019 raft_consensus.cc:2454] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d45599b92ba44e1aefd173322b03078 in term 1.
I20260809 08:51:13.006122 5224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cb9e5198857a41278343e686222dac12" candidate_uuid: "2d45599b92ba44e1aefd173322b03078" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75c07559595c443082801cf92e422169"
I20260809 08:51:13.006178 5325 leader_election.cc:302] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d45599b92ba44e1aefd173322b03078, 7a29060e6912440bb8ea849c263f62ce; no voters:
I20260809 08:51:13.006191 5224 raft_consensus.cc:3037] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.006278 5542 raft_consensus.cc:2781] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:13.006430 5542 raft_consensus.cc:686] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 1 LEADER]: Becoming Leader. State: Replica: 2d45599b92ba44e1aefd173322b03078, State: Running, Role: LEADER
I20260809 08:51:13.006520 5542 consensus_queue.cc:237] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } }
I20260809 08:51:13.007067 5224 raft_consensus.cc:2454] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2d45599b92ba44e1aefd173322b03078 in term 1.
I20260809 08:51:13.007326 4819 catalog_manager.cc:5495] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2d45599b92ba44e1aefd173322b03078 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "2d45599b92ba44e1aefd173322b03078" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2d45599b92ba44e1aefd173322b03078" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 35099 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 } health_report { overall_health: UNKNOWN } } }
I20260809 08:51:13.031610 5521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.029409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41282 (local address 127.10.184.67:35099)
0809 08:51:13.029454 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.029456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.029519 (+ 63us) server_negotiation.cc:407] Connection header received
0809 08:51:13.029593 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.029594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.029612 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.029668 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:13.029799 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.030225 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.030941 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.031118 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.031171 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.031325 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:13.031413 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:13.031480 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.031505 (+ 25us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.031524 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:51:13.033071 5224 raft_consensus.cc:1261] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 1 FOLLOWER]: Refusing update from remote peer 2d45599b92ba44e1aefd173322b03078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:13.033090 5019 raft_consensus.cc:1261] T cb9e5198857a41278343e686222dac12 P 7a29060e6912440bb8ea849c263f62ce [term 1 FOLLOWER]: Refusing update from remote peer 2d45599b92ba44e1aefd173322b03078: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:13.033406 5548 consensus_queue.cc:1035] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75c07559595c443082801cf92e422169" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:13.033541 5542 consensus_queue.cc:1035] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a29060e6912440bb8ea849c263f62ce" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 42591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:13.036396 5553 mvcc.cc:205] Tried to move back new op lower bound from 7316543377541419008 to 7316543377435373568. Current Snapshot: MvccSnapshot[applied={T|T < 7316543377541419008 or (T in {7316543377541419008})}]
I20260809 08:51:13.038094 4819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:13.038199 4819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:13.038872 4819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79ede5ae2093431ebec1e730f685dc60: Sending DeleteTablet for 3 replicas of tablet cb9e5198857a41278343e686222dac12
I20260809 08:51:13.039227 5196 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9e5198857a41278343e686222dac12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:49552
I20260809 08:51:13.039240 5400 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9e5198857a41278343e686222dac12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:41270
I20260809 08:51:13.039305 4989 tablet_service.cc:1514] Processing DeleteTablet for tablet cb9e5198857a41278343e686222dac12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:37046
I20260809 08:51:13.039599 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 4903
I20260809 08:51:13.059091 5567 tablet_replica.cc:330] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: stopping tablet replica
I20260809 08:51:13.059245 5567 raft_consensus.cc:2229] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:13.059376 5567 raft_consensus.cc:2258] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:13.059854 5567 ts_tablet_manager.cc:1905] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:13.061041 5566 tablet_replica.cc:330] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: stopping tablet replica
I20260809 08:51:13.061218 5566 raft_consensus.cc:2229] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:51:13.061408 5566 raft_consensus.cc:2258] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:13.061429 5567 ts_tablet_manager.cc:1918] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:13.061491 5567 log.cc:1196] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/cb9e5198857a41278343e686222dac12
I20260809 08:51:13.061924 5567 ts_tablet_manager.cc:1939] T cb9e5198857a41278343e686222dac12 P 75c07559595c443082801cf92e422169: Deleting consensus metadata
I20260809 08:51:13.061975 5566 ts_tablet_manager.cc:1905] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:13.062521 4800 catalog_manager.cc:4841] TS 75c07559595c443082801cf92e422169 (127.10.184.66:43195): tablet cb9e5198857a41278343e686222dac12 (table test-table [id=419659ceb6504b0dafa203cd4f293271]) successfully deleted
I20260809 08:51:13.063459 5566 ts_tablet_manager.cc:1918] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:13.063535 5566 log.cc:1196] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/cb9e5198857a41278343e686222dac12
I20260809 08:51:13.063758 5566 ts_tablet_manager.cc:1939] T cb9e5198857a41278343e686222dac12 P 2d45599b92ba44e1aefd173322b03078: Deleting consensus metadata
I20260809 08:51:13.064137 4802 catalog_manager.cc:4841] TS 2d45599b92ba44e1aefd173322b03078 (127.10.184.67:35099): tablet cb9e5198857a41278343e686222dac12 (table test-table [id=419659ceb6504b0dafa203cd4f293271]) successfully deleted
W20260809 08:51:13.064620 4802 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:42591 (error 108)
I20260809 08:51:13.064857 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 5106
I20260809 08:51:13.067935 5520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.065049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:42591 (local address 127.0.0.1:37062)
0809 08:51:13.067863 (+ 2814us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.067908 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:42591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2792}
W20260809 08:51:13.068071 4802 catalog_manager.cc:4584] TS 7a29060e6912440bb8ea849c263f62ce (127.10.184.65:42591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cb9e5198857a41278343e686222dac12: Network error: Client connection negotiation failed: client connection to 127.10.184.65:42591: connect: Connection refused (error 111)
I20260809 08:51:13.078111 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 5299
I20260809 08:51:13.086333 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 4775
2026-08-09T08:51:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2043 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T08:51:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:13Z Disabled control of system clock
I20260809 08:51:13.117192 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:46795
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:45787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:46795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:13.196470 5575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:13.196689 5575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:13.196709 5575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:13.197847 5575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:13.197889 5575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:13.197901 5575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:13.197913 5575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:13.197937 5575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:13.199244 5575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:46795
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:46795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:13.199468 5575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:13.199733 5575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:13.204442 5587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.204411 5590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.204397 5581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:13.204459 5575 server_base.cc:1034] running on GCE node
I20260809 08:51:13.204798 5575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:13.205085 5575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:13.206225 5575 hybrid_clock.cc:648] HybridClock initialized: now 1786265473206212 us; error 28 us; skew 500 ppm
I20260809 08:51:13.207396 5575 webserver.cc:460] Webserver started at http://127.10.184.126:43533/ using document root <none> and password file <none>
I20260809 08:51:13.207574 5575 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:13.207613 5575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:13.207715 5575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:13.208649 5575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "815cd63b49c746da83ee256cc99dd15d"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "90c671797e74ff0491a502c6e1574291"
server_key_iv: "2e5a08cd463c386b60a3cf64a23017cd"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.209007 5575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "815cd63b49c746da83ee256cc99dd15d"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "90c671797e74ff0491a502c6e1574291"
server_key_iv: "2e5a08cd463c386b60a3cf64a23017cd"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.210145 5575 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:13.210770 5596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.210955 5575 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:13.211035 5575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "815cd63b49c746da83ee256cc99dd15d"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "90c671797e74ff0491a502c6e1574291"
server_key_iv: "2e5a08cd463c386b60a3cf64a23017cd"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.211090 5575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:13.226423 5575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:13.226655 5575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:13.226760 5575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:13.230437 5575 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:46795
I20260809 08:51:13.230475 5676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:46795 every 8 connection(s)
I20260809 08:51:13.230769 5575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:13.231212 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 5575
I20260809 08:51:13.231315 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:13.231488 10977 external_mini_cluster.cc:1428] Setting key baec5b53545ed52ebb8f28eccb7d68bb
I20260809 08:51:13.231779 5677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:13.235051 5677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: Bootstrap starting.
I20260809 08:51:13.235683 5677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:13.235978 5677 log.cc:824] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:13.236786 5677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: No bootstrap required, opened a new log
I20260809 08:51:13.237498 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.231967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37306 (local address 127.10.184.126:46795)
0809 08:51:13.232194 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.232202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.232236 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:51:13.232836 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.232845 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.232954 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.233102 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:13.233283 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.234583 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.235439 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.236027 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.236158 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.236812 (+ 654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:13.236817 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:13.236817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:13.237025 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:13.237067 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:13.237081 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:13.237157 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.237212 (+ 55us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.237292 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
I20260809 08:51:13.238857 5677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } }
I20260809 08:51:13.238984 5677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:13.239005 5677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 815cd63b49c746da83ee256cc99dd15d, State: Initialized, Role: FOLLOWER
I20260809 08:51:13.239089 5677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } }
I20260809 08:51:13.239137 5677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:13.239158 5677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:13.239187 5677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.239641 5677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } }
I20260809 08:51:13.239735 5677 leader_election.cc:302] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 815cd63b49c746da83ee256cc99dd15d; no voters:
I20260809 08:51:13.239864 5677 leader_election.cc:288] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:13.239917 5690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:13.240083 5677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:13.240082 5690 raft_consensus.cc:686] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [term 1 LEADER]: Becoming Leader. State: Replica: 815cd63b49c746da83ee256cc99dd15d, State: Running, Role: LEADER
I20260809 08:51:13.240222 5690 consensus_queue.cc:237] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } }
I20260809 08:51:13.240564 5692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 815cd63b49c746da83ee256cc99dd15d. Latest consensus state: current_term: 1 leader_uuid: "815cd63b49c746da83ee256cc99dd15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } } }
I20260809 08:51:13.240564 5691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "815cd63b49c746da83ee256cc99dd15d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "815cd63b49c746da83ee256cc99dd15d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46795 } } }
I20260809 08:51:13.240736 5692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:13.240748 5691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d [sys.catalog]: This master's current role is: LEADER
W20260809 08:51:13.241741 5711 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:13.241787 5711 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:13.241844 5701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:13.242038 5701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:13.243402 5701 catalog_manager.cc:1347] Generated new cluster ID: bd1845429da44c4b913773ec0ac2030c
I20260809 08:51:13.243456 5701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:13.259541 5701 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:13.260191 5701 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:13.269588 5701 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: Generated new TSK 0
I20260809 08:51:13.269757 5701 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:13.271615 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--builtin_ntp_servers=127.10.184.84:45787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:13.370388 5715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:13.370570 5715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:13.370589 5715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:13.371711 5715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:13.371750 5715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:13.371773 5715 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:13.373123 5715 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:13.373344 5715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:13.373622 5715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:13.378328 5722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.378434 5723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.378324 5727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:13.378913 5715 server_base.cc:1034] running on GCE node
I20260809 08:51:13.379113 5715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:13.379341 5715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:13.380481 5715 hybrid_clock.cc:648] HybridClock initialized: now 1786265473380467 us; error 31 us; skew 500 ppm
I20260809 08:51:13.381743 5715 webserver.cc:460] Webserver started at http://127.10.184.65:38091/ using document root <none> and password file <none>
I20260809 08:51:13.381968 5715 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:13.382015 5715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:13.382138 5715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:13.383117 5715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "71976ca5eca441aa8669e5cf382eabb1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "94580cd48632ec327b8ef2e543fb76f4"
server_key_iv: "924ee6c832f9f9e6fb4fe7894aa5014d"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.383446 5715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "71976ca5eca441aa8669e5cf382eabb1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "94580cd48632ec327b8ef2e543fb76f4"
server_key_iv: "924ee6c832f9f9e6fb4fe7894aa5014d"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.384661 5715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:13.385423 5735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.385591 5715 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.385679 5715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "71976ca5eca441aa8669e5cf382eabb1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "94580cd48632ec327b8ef2e543fb76f4"
server_key_iv: "924ee6c832f9f9e6fb4fe7894aa5014d"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.385749 5715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:13.399128 5715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:13.399385 5715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:13.399507 5715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:13.399698 5715 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:13.399987 5715 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:13.400019 5715 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.400045 5715 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:13.400069 5715 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.405558 5715 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:43465
I20260809 08:51:13.405594 5913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:43465 every 8 connection(s)
I20260809 08:51:13.405892 5715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:51:13.407395 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 5715
I20260809 08:51:13.407492 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:13.407655 10977 external_mini_cluster.cc:1428] Setting key be7226feac18c61851a4d8cf69d15cde
I20260809 08:51:13.408537 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--builtin_ntp_servers=127.10.184.84:45787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:51:13.410956 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.406541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:37113 (local address 127.10.184.126:46795)
0809 08:51:13.406598 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.406601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.406809 (+ 208us) server_negotiation.cc:407] Connection header received
0809 08:51:13.407305 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.407307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.407322 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.407374 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:13.408126 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.408614 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.409647 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.409839 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.409892 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.410723 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:13.410724 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:13.410725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:13.410729 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:13.410745 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:13.410749 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:13.410784 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.410869 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.410900 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 08:51:13.411238 5918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.406529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:46795 (local address 127.10.184.65:37113)
0809 08:51:13.406701 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.406733 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.407199 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.407434 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.407437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.407445 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:13.408043 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.408049 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.408716 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.408717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.409553 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.409556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.409907 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.409908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.410064 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.410573 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:13.410574 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:13.410612 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:13.410640 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:13.410798 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:13.410803 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:13.410805 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.410871 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.410973 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260809 08:51:13.411677 5914 heartbeater.cc:347] Connected to a master server at 127.10.184.126:46795
I20260809 08:51:13.411768 5914 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:13.411940 5914 heartbeater.cc:511] Master 127.10.184.126:46795 requested a full tablet report, sending...
I20260809 08:51:13.412402 5623 ts_manager.cc:194] Registered new tserver with Master: 71976ca5eca441aa8669e5cf382eabb1 (127.10.184.65:43465)
I20260809 08:51:13.413055 5623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.65:37113
W20260809 08:51:13.490895 5919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:13.491088 5919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:13.491117 5919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:13.492213 5919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:13.492251 5919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:13.492283 5919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:13.493639 5919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5919
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:13.493857 5919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:13.494118 5919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:13.498878 5932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:13.498968 5919 server_base.cc:1034] running on GCE node
W20260809 08:51:13.498894 5934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.498894 5928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:13.499339 5919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:13.499553 5919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:13.500684 5919 hybrid_clock.cc:648] HybridClock initialized: now 1786265473500674 us; error 27 us; skew 500 ppm
I20260809 08:51:13.501830 5919 webserver.cc:460] Webserver started at http://127.10.184.66:45409/ using document root <none> and password file <none>
I20260809 08:51:13.502036 5919 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:13.502079 5919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:13.502229 5919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:13.503243 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "89ad1642ff8246a68e9d6f6dea2fddcb"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ce0ff00bf27ca7639122cdde06ecadee"
server_key_iv: "41e58346bee0e680d16c90cdfc5d949b"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.503567 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "89ad1642ff8246a68e9d6f6dea2fddcb"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ce0ff00bf27ca7639122cdde06ecadee"
server_key_iv: "41e58346bee0e680d16c90cdfc5d949b"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.504704 5919 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:13.505406 5944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.505548 5919 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.505625 5919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "89ad1642ff8246a68e9d6f6dea2fddcb"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ce0ff00bf27ca7639122cdde06ecadee"
server_key_iv: "41e58346bee0e680d16c90cdfc5d949b"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.505692 5919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:13.513701 5919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:13.513939 5919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:13.514055 5919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:13.514256 5919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:13.514533 5919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:13.514565 5919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.514585 5919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:13.514607 5919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.519897 5919 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:34145
I20260809 08:51:13.519933 6127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:34145 every 8 connection(s)
I20260809 08:51:13.520211 5919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:13.523836 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 5919
I20260809 08:51:13.523936 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:13.524129 10977 external_mini_cluster.cc:1428] Setting key e425da21d8568d49bb08e7f42cc687c4
I20260809 08:51:13.525143 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--builtin_ntp_servers=127.10.184.84:45787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:51:13.525550 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.520833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:51767 (local address 127.10.184.126:46795)
0809 08:51:13.520913 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.520915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.521095 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:51:13.521554 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.521556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.521567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.521618 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:13.522258 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.522994 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.524196 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.524485 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.524538 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.525339 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:13.525340 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:13.525341 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:13.525345 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:13.525362 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:13.525366 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:13.525407 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.525472 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.525494 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:51:13.525785 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.520801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:46795 (local address 127.10.184.66:51767)
0809 08:51:13.520992 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.521023 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.521463 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.521690 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.521693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.521703 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:13.522160 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.522165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.523095 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.523096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.524111 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.524114 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.524560 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.524561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.524715 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.525193 (+ 478us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:13.525194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:13.525232 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:13.525261 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:13.525409 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:13.525413 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:13.525416 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.525474 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.525569 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":104,"threads_started":1}
I20260809 08:51:13.526226 6128 heartbeater.cc:347] Connected to a master server at 127.10.184.126:46795
I20260809 08:51:13.526309 6128 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:13.526472 6128 heartbeater.cc:511] Master 127.10.184.126:46795 requested a full tablet report, sending...
I20260809 08:51:13.526801 5623 ts_manager.cc:194] Registered new tserver with Master: 89ad1642ff8246a68e9d6f6dea2fddcb (127.10.184.66:34145)
I20260809 08:51:13.527146 5623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.66:51767
W20260809 08:51:13.604727 6133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:13.604957 6133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:13.604986 6133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:13.606102 6133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:13.606137 6133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:13.606165 6133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:13.607470 6133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:46795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6133
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:13.607690 6133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:13.607980 6133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:13.612715 6142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.612757 6141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:13.612732 6146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:13.612862 6133 server_base.cc:1034] running on GCE node
I20260809 08:51:13.613086 6133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:13.613322 6133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:13.614452 6133 hybrid_clock.cc:648] HybridClock initialized: now 1786265473614437 us; error 29 us; skew 500 ppm
I20260809 08:51:13.615607 6133 webserver.cc:460] Webserver started at http://127.10.184.67:41725/ using document root <none> and password file <none>
I20260809 08:51:13.615839 6133 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:13.615887 6133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:13.616008 6133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:13.617038 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "85f21ab754e742f284dda6610fd573a1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ddf71d6976b259a1418c3b1110ffc1f7"
server_key_iv: "bb83caae8fe3f178315f37924127c9df"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.617357 6133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "85f21ab754e742f284dda6610fd573a1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ddf71d6976b259a1418c3b1110ffc1f7"
server_key_iv: "bb83caae8fe3f178315f37924127c9df"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.618498 6133 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:51:13.619107 6153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.619237 6133 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.619320 6133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "85f21ab754e742f284dda6610fd573a1"
format_stamp: "Formatted at 2026-08-09 08:51:13 on dist-test-slave-ncp9"
server_key: "ddf71d6976b259a1418c3b1110ffc1f7"
server_key_iv: "bb83caae8fe3f178315f37924127c9df"
server_key_version: "encryptionkey@0"
I20260809 08:51:13.619375 6133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:13.630700 6133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:13.630939 6133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:13.631060 6133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:13.631251 6133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:13.631533 6133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:13.631567 6133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.631592 6133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:13.631618 6133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:13.637014 6133 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:34069
I20260809 08:51:13.637074 6326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:34069 every 8 connection(s)
I20260809 08:51:13.637351 6133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:51:13.640336 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 6133
I20260809 08:51:13.640465 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:13.640659 10977 external_mini_cluster.cc:1428] Setting key f7dd37435c98738b6ba6113b3ad5ebdd
I20260809 08:51:13.642776 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.638011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:44707 (local address 127.10.184.126:46795)
0809 08:51:13.638071 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.638073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.638290 (+ 217us) server_negotiation.cc:407] Connection header received
0809 08:51:13.638793 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.638795 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.638808 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.638861 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:13.639673 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.640187 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.641448 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.641686 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.641744 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.642511 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:13.642513 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:13.642513 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:13.642518 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:13.642534 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:13.642538 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:13.642582 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.642672 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.642698 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:51:13.642969 6332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.638009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:46795 (local address 127.10.184.67:44707)
0809 08:51:13.638181 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.638212 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.638694 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.638924 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.638927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.638935 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:13.639567 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.639574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.640281 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.640282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.641378 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.641381 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.641750 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.641752 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.641885 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.642383 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:13.642384 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:13.642413 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:13.642435 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:13.642591 (+ 156us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:13.642596 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:13.642599 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.642672 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.642768 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 08:51:13.643352 6327 heartbeater.cc:347] Connected to a master server at 127.10.184.126:46795
I20260809 08:51:13.643427 6327 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:13.643586 6327 heartbeater.cc:511] Master 127.10.184.126:46795 requested a full tablet report, sending...
I20260809 08:51:13.643911 5623 ts_manager.cc:194] Registered new tserver with Master: 85f21ab754e742f284dda6610fd573a1 (127.10.184.67:34069)
I20260809 08:51:13.644284 5623 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.67:44707
I20260809 08:51:13.652431 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:13.656528 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.653803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.10.184.126:46795)
0809 08:51:13.653855 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.653857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.653980 (+ 123us) server_negotiation.cc:407] Connection header received
0809 08:51:13.654050 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.654051 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.654062 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.654117 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:13.654262 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.654746 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.655527 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.655710 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.655770 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.656313 (+ 543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:13.656314 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:13.656315 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:13.656319 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:13.656333 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:13.656337 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:13.656381 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.656435 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.656455 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:51:13.658043 5623 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:13.659580 5623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:13.665709 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40500 (local address 127.10.184.67:34069)
0809 08:51:13.662133 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.662137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.662254 (+ 117us) server_negotiation.cc:407] Connection header received
0809 08:51:13.662369 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.662372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.662385 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.662466 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.662629 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.663363 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.664770 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.665344 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665415 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.665497 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.665594 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.665628 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260809 08:51:13.665711 6346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34069 (local address 127.0.0.1:40500)
0809 08:51:13.662195 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.662203 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.662279 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.662520 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.662521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.662523 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.662557 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.662558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.663436 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.663437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.664698 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.664700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.665422 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665422 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.665495 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.665509 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.665581 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.665635 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":2,"thread_start_us":70,"threads_started":1}
I20260809 08:51:13.666196 6344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:34145 (local address 127.0.0.1:53502)
0809 08:51:13.661883 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.661925 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.662010 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.662282 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.662283 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.662286 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.662331 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.662332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.663575 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.663576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.665086 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.665088 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.665971 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.666032 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.666036 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.666089 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.666129 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":358,"thread_start_us":71,"threads_started":1}
I20260809 08:51:13.666234 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53502 (local address 127.10.184.66:34145)
0809 08:51:13.661939 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.661943 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.662079 (+ 136us) server_negotiation.cc:407] Connection header received
0809 08:51:13.662124 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.662126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.662141 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.662219 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.662395 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.663494 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665178 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.665891 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665957 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.666041 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.666101 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.666143 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260809 08:51:13.666520 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43465 (local address 127.0.0.1:41108)
0809 08:51:13.661925 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.661935 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.662005 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.662212 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.662217 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.662219 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.662283 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.662285 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.664466 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.664467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.665369 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.665371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.666302 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.666303 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.666362 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.666365 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.666407 (+ 42us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.666456 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":83,"threads_started":1}
I20260809 08:51:13.666551 6228 tablet_service.cc:1467] Processing CreateTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 (DEFAULT_TABLE table=test-table [id=bdf8566b25cc48269c6fc4d0d02cca01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:13.666779 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.661709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41108 (local address 127.10.184.65:43465)
0809 08:51:13.661853 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.661858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.661995 (+ 137us) server_negotiation.cc:407] Connection header received
0809 08:51:13.662092 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.662094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.662109 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.662198 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.662367 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.663496 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.665460 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.666127 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.666184 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.666265 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.666658 (+ 393us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.666701 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":83,"threads_started":1}
I20260809 08:51:13.666931 6228 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776acfc95a2f4a6c80168bc9e3c15da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:13.667088 6014 tablet_service.cc:1467] Processing CreateTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 (DEFAULT_TABLE table=test-table [id=bdf8566b25cc48269c6fc4d0d02cca01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:13.667341 6014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776acfc95a2f4a6c80168bc9e3c15da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:13.669327 6358 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Bootstrap starting.
I20260809 08:51:13.669804 6359 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Bootstrap starting.
I20260809 08:51:13.670030 6358 tablet_bootstrap.cc:654] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:13.670300 6358 log.cc:824] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:13.670463 6359 tablet_bootstrap.cc:654] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:13.670380 5811 tablet_service.cc:1467] Processing CreateTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 (DEFAULT_TABLE table=test-table [id=bdf8566b25cc48269c6fc4d0d02cca01]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:13.670717 5811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 776acfc95a2f4a6c80168bc9e3c15da3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:13.670816 6359 log.cc:824] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:13.670917 6358 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: No bootstrap required, opened a new log
I20260809 08:51:13.670962 6358 ts_tablet_manager.cc:1397] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:13.671538 6359 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: No bootstrap required, opened a new log
I20260809 08:51:13.671594 6359 ts_tablet_manager.cc:1397] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:13.672760 6358 raft_consensus.cc:348] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.672880 6358 raft_consensus.cc:374] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:13.672935 6358 raft_consensus.cc:729] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85f21ab754e742f284dda6610fd573a1, State: Initialized, Role: FOLLOWER
I20260809 08:51:13.673062 6358 consensus_queue.cc:260] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.673388 6358 ts_tablet_manager.cc:1428] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 08:51:13.673424 6327 heartbeater.cc:503] Master 127.10.184.126:46795 was elected leader, sending a full tablet report...
I20260809 08:51:13.674257 6359 raft_consensus.cc:348] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.674373 6363 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: Bootstrap starting.
I20260809 08:51:13.674482 6359 raft_consensus.cc:374] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:13.674536 6359 raft_consensus.cc:729] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ad1642ff8246a68e9d6f6dea2fddcb, State: Initialized, Role: FOLLOWER
I20260809 08:51:13.674628 6359 consensus_queue.cc:260] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.674863 6359 ts_tablet_manager.cc:1428] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:51:13.674886 6128 heartbeater.cc:503] Master 127.10.184.126:46795 was elected leader, sending a full tablet report...
I20260809 08:51:13.674973 6363 tablet_bootstrap.cc:654] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:13.675266 6363 log.cc:824] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:13.675948 6363 tablet_bootstrap.cc:492] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: No bootstrap required, opened a new log
I20260809 08:51:13.676002 6363 ts_tablet_manager.cc:1397] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:13.677842 6363 raft_consensus.cc:348] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.677963 6363 raft_consensus.cc:374] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:13.677983 6363 raft_consensus.cc:729] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71976ca5eca441aa8669e5cf382eabb1, State: Initialized, Role: FOLLOWER
I20260809 08:51:13.678087 6363 consensus_queue.cc:260] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.678311 6363 ts_tablet_manager.cc:1428] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:51:13.678357 5914 heartbeater.cc:503] Master 127.10.184.126:46795 was elected leader, sending a full tablet report...
I20260809 08:51:13.732064 6364 raft_consensus.cc:482] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:13.732203 6364 raft_consensus.cc:504] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.732540 6364 leader_election.cc:288] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 71976ca5eca441aa8669e5cf382eabb1 (127.10.184.65:43465), 85f21ab754e742f284dda6610fd573a1 (127.10.184.67:34069)
I20260809 08:51:13.735265 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.732768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:50813 (local address 127.10.184.65:43465)
0809 08:51:13.732827 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.732829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.732837 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:13.732860 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.732861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.732870 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.732952 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.733112 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.733674 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.734594 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.734959 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.735019 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.735028 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.735156 (+ 128us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.735192 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:51:13.735283 6131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.732691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43465 (local address 127.10.184.66:50813)
0809 08:51:13.732733 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.732744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.732782 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.732962 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.732963 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.732965 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.733040 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.733042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.733748 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.733750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.734508 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.734510 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.735030 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.735031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.735095 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.735098 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.735142 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.735192 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":22}
I20260809 08:51:13.735318 6356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.732768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:48723 (local address 127.10.184.67:34069)
0809 08:51:13.732870 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.732872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.732988 (+ 116us) server_negotiation.cc:407] Connection header received
0809 08:51:13.733068 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.733069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.733078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.733126 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:13.733305 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.733785 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.734675 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.735017 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.735074 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.735082 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.735222 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.735256 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 08:51:13.735358 6367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.732691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:34069 (local address 127.10.184.66:48723)
0809 08:51:13.732927 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.732935 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:13.732982 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:13.733170 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:13.733170 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:13.733172 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:13.733246 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.733248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.733856 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.733857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.734589 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:13.734591 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.735086 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.735087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:13.735158 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.735160 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:51:13.735204 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 08:51:13.735242 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":46,"thread_start_us":111,"threads_started":1}
I20260809 08:51:13.735698 6259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776acfc95a2f4a6c80168bc9e3c15da3" candidate_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85f21ab754e742f284dda6610fd573a1" is_pre_election: true
I20260809 08:51:13.735827 6259 raft_consensus.cc:2454] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ad1642ff8246a68e9d6f6dea2fddcb in term 0.
I20260809 08:51:13.735812 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776acfc95a2f4a6c80168bc9e3c15da3" candidate_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71976ca5eca441aa8669e5cf382eabb1" is_pre_election: true
I20260809 08:51:13.735956 5843 raft_consensus.cc:2454] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 89ad1642ff8246a68e9d6f6dea2fddcb in term 0.
I20260809 08:51:13.736047 5945 leader_election.cc:302] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 85f21ab754e742f284dda6610fd573a1, 89ad1642ff8246a68e9d6f6dea2fddcb; no voters:
I20260809 08:51:13.736176 6364 raft_consensus.cc:2781] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:13.736233 6364 raft_consensus.cc:482] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:13.736255 6364 raft_consensus.cc:3037] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.736913 6364 raft_consensus.cc:504] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.737016 6364 leader_election.cc:288] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [CANDIDATE]: Term 1 election: Requested vote from peers 71976ca5eca441aa8669e5cf382eabb1 (127.10.184.65:43465), 85f21ab754e742f284dda6610fd573a1 (127.10.184.67:34069)
I20260809 08:51:13.737161 5843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776acfc95a2f4a6c80168bc9e3c15da3" candidate_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "71976ca5eca441aa8669e5cf382eabb1"
I20260809 08:51:13.737186 6259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "776acfc95a2f4a6c80168bc9e3c15da3" candidate_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85f21ab754e742f284dda6610fd573a1"
I20260809 08:51:13.737248 6259 raft_consensus.cc:3037] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.737246 5843 raft_consensus.cc:3037] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:13.738097 5843 raft_consensus.cc:2454] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ad1642ff8246a68e9d6f6dea2fddcb in term 1.
I20260809 08:51:13.738097 6259 raft_consensus.cc:2454] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89ad1642ff8246a68e9d6f6dea2fddcb in term 1.
I20260809 08:51:13.738272 5949 leader_election.cc:302] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 71976ca5eca441aa8669e5cf382eabb1, 89ad1642ff8246a68e9d6f6dea2fddcb; no voters:
I20260809 08:51:13.738377 6364 raft_consensus.cc:2781] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:13.738494 6364 raft_consensus.cc:686] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 1 LEADER]: Becoming Leader. State: Replica: 89ad1642ff8246a68e9d6f6dea2fddcb, State: Running, Role: LEADER
I20260809 08:51:13.738595 6364 consensus_queue.cc:237] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } }
I20260809 08:51:13.739349 5623 catalog_manager.cc:5495] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 89ad1642ff8246a68e9d6f6dea2fddcb (127.10.184.66). New cstate: current_term: 1 leader_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89ad1642ff8246a68e9d6f6dea2fddcb" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 34145 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 } health_report { overall_health: UNKNOWN } } }
I20260809 08:51:13.755447 6348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.753381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53516 (local address 127.10.184.66:34145)
0809 08:51:13.753419 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:13.753420 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:13.753443 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:51:13.753505 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:13.753505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:13.753520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:13.753573 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:13.753664 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.754095 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.754759 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:13.754932 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:13.754979 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:13.755155 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:13.755246 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:13.755317 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:13.755355 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 08:51:13.755374 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:51:13.756982 6259 raft_consensus.cc:1261] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 1 FOLLOWER]: Refusing update from remote peer 89ad1642ff8246a68e9d6f6dea2fddcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:13.757012 5843 raft_consensus.cc:1261] T 776acfc95a2f4a6c80168bc9e3c15da3 P 71976ca5eca441aa8669e5cf382eabb1 [term 1 FOLLOWER]: Refusing update from remote peer 89ad1642ff8246a68e9d6f6dea2fddcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:13.757277 6368 consensus_queue.cc:1035] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "85f21ab754e742f284dda6610fd573a1" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 34069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:13.757391 6364 consensus_queue.cc:1035] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "71976ca5eca441aa8669e5cf382eabb1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 43465 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:13.761812 5622 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:13.761905 5622 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:13.762611 5622 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 815cd63b49c746da83ee256cc99dd15d: Sending DeleteTablet for 3 replicas of tablet 776acfc95a2f4a6c80168bc9e3c15da3
I20260809 08:51:13.762903 6014 tablet_service.cc:1514] Processing DeleteTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='slave'} at 127.0.0.1:53502
I20260809 08:51:13.762951 5811 tablet_service.cc:1514] Processing DeleteTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='slave'} at 127.0.0.1:41108
I20260809 08:51:13.762979 6228 tablet_service.cc:1514] Processing DeleteTablet for tablet 776acfc95a2f4a6c80168bc9e3c15da3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:13 UTC) from {username='slave'} at 127.0.0.1:40500
I20260809 08:51:13.763118 6392 tablet_replica.cc:330] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: stopping tablet replica
I20260809 08:51:13.763211 6392 raft_consensus.cc:2229] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:51:13.763324 6392 raft_consensus.cc:2258] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:13.763345 6393 tablet_replica.cc:330] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: stopping tablet replica
I20260809 08:51:13.763432 6393 raft_consensus.cc:2229] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:13.763430 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 5715
I20260809 08:51:13.763536 6393 raft_consensus.cc:2258] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:13.764273 6392 ts_tablet_manager.cc:1905] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:13.766203 6392 ts_tablet_manager.cc:1918] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:13.766338 6392 log.cc:1196] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/776acfc95a2f4a6c80168bc9e3c15da3
I20260809 08:51:13.766710 6392 ts_tablet_manager.cc:1939] T 776acfc95a2f4a6c80168bc9e3c15da3 P 89ad1642ff8246a68e9d6f6dea2fddcb: Deleting consensus metadata
I20260809 08:51:13.767194 5598 catalog_manager.cc:4841] TS 89ad1642ff8246a68e9d6f6dea2fddcb (127.10.184.66:34145): tablet 776acfc95a2f4a6c80168bc9e3c15da3 (table test-table [id=bdf8566b25cc48269c6fc4d0d02cca01]) successfully deleted
I20260809 08:51:13.785081 6393 ts_tablet_manager.cc:1905] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:13.786465 6393 ts_tablet_manager.cc:1918] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:13.786537 6393 log.cc:1196] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/776acfc95a2f4a6c80168bc9e3c15da3
I20260809 08:51:13.786754 6393 ts_tablet_manager.cc:1939] T 776acfc95a2f4a6c80168bc9e3c15da3 P 85f21ab754e742f284dda6610fd573a1: Deleting consensus metadata
I20260809 08:51:13.787151 5598 catalog_manager.cc:4841] TS 85f21ab754e742f284dda6610fd573a1 (127.10.184.67:34069): tablet 776acfc95a2f4a6c80168bc9e3c15da3 (table test-table [id=bdf8566b25cc48269c6fc4d0d02cca01]) successfully deleted
W20260809 08:51:13.790104 5605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:43465 (error 108)
I20260809 08:51:13.790588 6352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:13.790471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:43465 (local address 127.0.0.1:41124)
0809 08:51:13.790519 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:13.790560 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:43465: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260809 08:51:13.790709 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 5919
W20260809 08:51:13.790706 5605 catalog_manager.cc:4584] TS 71976ca5eca441aa8669e5cf382eabb1 (127.10.184.65:43465): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 776acfc95a2f4a6c80168bc9e3c15da3: Network error: Client connection negotiation failed: client connection to 127.10.184.65:43465: connect: Connection refused (error 111)
I20260809 08:51:13.797489 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 6133
I20260809 08:51:13.804550 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 5575
2026-08-09T08:51:13Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (716 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:13 dist-test-slave-ncp9 krb5kdc[6399](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:13 dist-test-slave-ncp9 krb5kdc[6399](info): set up 2 sockets
Aug 09 08:51:13 dist-test-slave-ncp9 krb5kdc[6399](info): commencing operation
krb5kdc: starting...
W20260809 08:51:14.952868 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:15.061632 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37619
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:37619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:15.139993 6418 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:15.140180 6418 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:15.140200 6418 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:15.141316 6418 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:15.141355 6418 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:15.141368 6418 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:15.141381 6418 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:15.141407 6418 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:15.142876 6418 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:37619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6418
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:15.143098 6418 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:15.143301 6418 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:15.147979 6427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.147998 6425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.147997 6429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.148239 6418 server_base.cc:1034] running on GCE node
I20260809 08:51:15.148488 6418 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:15.148770 6418 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:15.149910 6418 hybrid_clock.cc:648] HybridClock initialized: now 1786265475149889 us; error 32 us; skew 500 ppm
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:15.152144 6418 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:15.152567 6418 webserver.cc:460] Webserver started at http://127.10.184.126:40087/ using document root <none> and password file <none>
I20260809 08:51:15.152758 6418 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:15.152803 6418 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:15.152951 6418 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:15.153923 6418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "67ebe511a1164160b534e181f1c7880d"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "c17f84f0559fbed645fc06cac78abc10"
server_key_iv: "07305dc95ebdd8999a3a1c73783b01f5"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.154246 6418 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "67ebe511a1164160b534e181f1c7880d"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "c17f84f0559fbed645fc06cac78abc10"
server_key_iv: "07305dc95ebdd8999a3a1c73783b01f5"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.155416 6418 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:15.156020 6436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.156169 6418 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:15.156252 6418 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "67ebe511a1164160b534e181f1c7880d"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "c17f84f0559fbed645fc06cac78abc10"
server_key_iv: "07305dc95ebdd8999a3a1c73783b01f5"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.156312 6418 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:15.168623 6418 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:15.170193 6418 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:15.170315 6418 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:15.173954 6418 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:37619
I20260809 08:51:15.173985 6533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:37619 every 8 connection(s)
I20260809 08:51:15.174276 6418 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:15.175058 6535 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:15.175309 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 6418
I20260809 08:51:15.175415 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:15.175575 10977 external_mini_cluster.cc:1428] Setting key eb55aeda7fb594fc6fd62ce0eda0963a
I20260809 08:51:15.177819 6535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: Bootstrap starting.
I20260809 08:51:15.178494 6535 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:15.178773 6535 log.cc:824] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:15.179387 6535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: No bootstrap required, opened a new log
I20260809 08:51:15.181604 6535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } }
I20260809 08:51:15.181777 6535 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:15.181823 6535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67ebe511a1164160b534e181f1c7880d, State: Initialized, Role: FOLLOWER
I20260809 08:51:15.181955 6535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } }
I20260809 08:51:15.182035 6535 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:15.182071 6535 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:15.182116 6535 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 0 FOLLOWER]: Advancing to term 1
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:15.182767 6535 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } }
I20260809 08:51:15.182880 6535 leader_election.cc:302] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67ebe511a1164160b534e181f1c7880d; no voters:
I20260809 08:51:15.183007 6535 leader_election.cc:288] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:15.183089 6542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:15.183238 6535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:15.183261 6542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [term 1 LEADER]: Becoming Leader. State: Replica: 67ebe511a1164160b534e181f1c7880d, State: Running, Role: LEADER
I20260809 08:51:15.183423 6542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } }
I20260809 08:51:15.183753 6544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67ebe511a1164160b534e181f1c7880d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } } }
I20260809 08:51:15.183987 6544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:15.183905 6543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67ebe511a1164160b534e181f1c7880d. Latest consensus state: current_term: 1 leader_uuid: "67ebe511a1164160b534e181f1c7880d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ebe511a1164160b534e181f1c7880d" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37619 } } }
I20260809 08:51:15.184237 6543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:15.184407 6556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:15.184975 6556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:15.186066 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.176037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48882 (local address 127.10.184.126:37619)
0809 08:51:15.176234 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.176243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.176278 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:51:15.176754 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.176763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.176820 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.176997 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.177567 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.178846 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.179574 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.180155 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.180284 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.182592 (+ 2308us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.182596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.182597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:15.182609 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.184549 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.184941 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.184944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.184945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.185054 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.185274 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.185275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.185275 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.185498 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.185629 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.185744 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.185868 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":82,"threads_started":1}
I20260809 08:51:15.186586 6556 catalog_manager.cc:1347] Generated new cluster ID: c9d76c6dcbea4771b819c16ff3c47937
I20260809 08:51:15.186635 6556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:15.198946 6556 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:15.199667 6556 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:15.210253 6556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: Generated new TSK 0
I20260809 08:51:15.210412 6556 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:51:15.239214 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:15.339092 6573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:15.339282 6573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:15.339304 6573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:15.340380 6573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:15.340415 6573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:15.340438 6573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:15.341857 6573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:15.342033 6573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:15.342227 6573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:15.347085 6579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.347065 6580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.347067 6582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.347379 6573 server_base.cc:1034] running on GCE node
I20260809 08:51:15.347573 6573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:15.347776 6573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:15.348933 6573 hybrid_clock.cc:648] HybridClock initialized: now 1786265475348916 us; error 30 us; skew 500 ppm
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:15.351240 6573 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:51:15.351639 6573 webserver.cc:460] Webserver started at http://127.10.184.65:44185/ using document root <none> and password file <none>
I20260809 08:51:15.351850 6573 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:15.351895 6573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:15.352016 6573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:15.353019 6573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "520707c858974d0899e3175ffc8fc99b"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "ac2f85fc1a920a0302710237befd93fb"
server_key_iv: "b7be8e2ca0f54a6d0343bf871f1661e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.353348 6573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "520707c858974d0899e3175ffc8fc99b"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "ac2f85fc1a920a0302710237befd93fb"
server_key_iv: "b7be8e2ca0f54a6d0343bf871f1661e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.354513 6573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:15.355146 6592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.355296 6573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:15.355378 6573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "520707c858974d0899e3175ffc8fc99b"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "ac2f85fc1a920a0302710237befd93fb"
server_key_iv: "b7be8e2ca0f54a6d0343bf871f1661e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.355436 6573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:15.364048 6573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:15.365633 6573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:15.365765 6573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:15.365952 6573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:15.366217 6573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:15.366250 6573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.366276 6573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:15.366293 6573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.371608 6573 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:39863
I20260809 08:51:15.371644 6763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:39863 every 8 connection(s)
I20260809 08:51:15.371912 6573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:51:15.373955 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 6573
I20260809 08:51:15.374066 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:15.374269 10977 external_mini_cluster.cc:1428] Setting key 8605afd630b82029285b281d94d7b9d1
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:15.381325 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.372540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:58465 (local address 127.10.184.126:37619)
0809 08:51:15.372591 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.372593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.372780 (+ 187us) server_negotiation.cc:407] Connection header received
0809 08:51:15.373243 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.373244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.373253 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.373302 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.374131 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.374820 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.376119 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.376336 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.376389 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.378623 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.378624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.378625 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:15.378632 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.379888 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.380348 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.380348 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.380349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.380392 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.380671 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.380672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.380672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.380789 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.380851 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.381192 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.381261 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:51:15.381568 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.372513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37619 (local address 127.10.184.65:58465)
0809 08:51:15.372668 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.372707 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.373169 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.373352 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.373355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.373870 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:15.374041 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.374046 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.374902 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.374904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.376037 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.376040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.376412 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.376413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.376555 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.377055 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:15.377063 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:15.378536 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:15.379969 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.379977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.379979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.380278 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.380444 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.380445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.380446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.380579 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.380858 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:15.380864 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:15.381042 (+ 178us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.381166 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.381342 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260809 08:51:15.382081 6764 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37619
I20260809 08:51:15.382184 6764 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:15.382352 6764 heartbeater.cc:511] Master 127.10.184.126:37619 requested a full tablet report, sending...
I20260809 08:51:15.382833 6459 ts_manager.cc:194] Registered new tserver with Master: 520707c858974d0899e3175ffc8fc99b (127.10.184.65:39863)
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
I20260809 08:51:15.383522 6459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:58465
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:51:15.397400 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:15.477618 6771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:15.477804 6771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:15.477833 6771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:15.478917 6771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:15.478955 6771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:15.478988 6771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:15.480381 6771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:15.480593 6771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:15.480803 6771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:15.485545 6780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.485558 6779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.485649 6771 server_base.cc:1034] running on GCE node
W20260809 08:51:15.485558 6782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.485916 6771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:15.486146 6771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:15.487285 6771 hybrid_clock.cc:648] HybridClock initialized: now 1786265475487268 us; error 28 us; skew 500 ppm
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:15.489530 6771 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:51:15.489945 6771 webserver.cc:460] Webserver started at http://127.10.184.66:37597/ using document root <none> and password file <none>
I20260809 08:51:15.490149 6771 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:15.490199 6771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:15.490316 6771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:15.491293 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "13a7d578f189441d8d968c490d258512"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "913f909da9d9f15268fea03cddb4c870"
server_key_iv: "3e3e1e0ae0ded9a2c8e978fa638b30e2"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.491607 6771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "13a7d578f189441d8d968c490d258512"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "913f909da9d9f15268fea03cddb4c870"
server_key_iv: "3e3e1e0ae0ded9a2c8e978fa638b30e2"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.492758 6771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:15.493438 6793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.493577 6771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.493659 6771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "13a7d578f189441d8d968c490d258512"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "913f909da9d9f15268fea03cddb4c870"
server_key_iv: "3e3e1e0ae0ded9a2c8e978fa638b30e2"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.493726 6771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:15.501021 6771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:15.502624 6771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:15.502764 6771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:15.502962 6771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:15.503232 6771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:15.503266 6771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.503291 6771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:15.503315 6771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.508615 6771 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:38365
I20260809 08:51:15.508670 6951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:38365 every 8 connection(s)
I20260809 08:51:15.508984 6771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:15.511394 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 6771
I20260809 08:51:15.511502 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:15.511668 10977 external_mini_cluster.cc:1428] Setting key bb15bab783f3db7842d48a16f79ee25a
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:15.517815 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.509622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:37393 (local address 127.10.184.126:37619)
0809 08:51:15.509698 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.509700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.509833 (+ 133us) server_negotiation.cc:407] Connection header received
0809 08:51:15.510280 (+ 447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.510281 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.510289 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.510336 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.511170 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.511695 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.512786 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.513076 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.513146 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.515263 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.515265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.515265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:15.515272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.516551 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.516943 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.516944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.516945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.516998 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.517224 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.517225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.517225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.517338 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.517393 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.517640 (+ 247us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.517728 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 08:51:15.517956 6955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.509595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37619 (local address 127.10.184.66:37393)
0809 08:51:15.509727 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.509761 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.510211 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.510387 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.510390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.510928 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:15.511106 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.511111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.511794 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.511795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.512707 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.512710 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.513148 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.513149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.513244 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.513678 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:15.513685 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:15.515176 (+ 1491us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:15.516626 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.516633 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.516635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.516846 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.517051 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.517052 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.517052 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.517167 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.517390 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:15.517397 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:15.517523 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.517610 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.517754 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260809 08:51:15.518362 6952 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37619
I20260809 08:51:15.518445 6952 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:15.518610 6952 heartbeater.cc:511] Master 127.10.184.126:37619 requested a full tablet report, sending...
I20260809 08:51:15.518989 6459 ts_manager.cc:194] Registered new tserver with Master: 13a7d578f189441d8d968c490d258512 (127.10.184.66:38365)
I20260809 08:51:15.519342 6459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:37393
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:51:15.535465 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:15.613642 6958 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:15.613828 6958 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:15.613850 6958 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:15.614948 6958 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:15.614985 6958 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:15.615011 6958 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:15.616403 6958 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:37619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6958
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:15.616617 6958 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:15.616829 6958 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:15.621615 6964 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.621718 6958 server_base.cc:1034] running on GCE node
W20260809 08:51:15.621644 6966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.621619 6970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.622023 6958 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:15.622260 6958 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:15.623399 6958 hybrid_clock.cc:648] HybridClock initialized: now 1786265475623383 us; error 29 us; skew 500 ppm
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:15.625818 6958 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:51:15.626185 6958 webserver.cc:460] Webserver started at http://127.10.184.67:37077/ using document root <none> and password file <none>
I20260809 08:51:15.626365 6958 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:15.626402 6958 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:15.626492 6958 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:15.627417 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "96a7ba8836434eda85c95dfb1d2dbfd8"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "de344d77d1a33f47008d1b8d1b010e72"
server_key_iv: "3b40cb484de9ee87510f7e54868d846d"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.627705 6958 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "96a7ba8836434eda85c95dfb1d2dbfd8"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "de344d77d1a33f47008d1b8d1b010e72"
server_key_iv: "3b40cb484de9ee87510f7e54868d846d"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.628824 6958 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:15.629498 6978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.629643 6958 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.629724 6958 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "96a7ba8836434eda85c95dfb1d2dbfd8"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "de344d77d1a33f47008d1b8d1b010e72"
server_key_iv: "3b40cb484de9ee87510f7e54868d846d"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.629776 6958 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:15.643891 6958 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:15.645574 6958 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:15.645679 6958 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:15.645838 6958 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:15.646067 6958 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:15.646095 6958 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.646113 6958 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:15.646128 6958 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.651278 6958 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:39611
I20260809 08:51:15.651345 7146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:39611 every 8 connection(s)
I20260809 08:51:15.651592 6958 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
Aug 09 08:51:15 dist-test-slave-ncp9 krb5kdc[6399](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265475, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:15.659655 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 6958
I20260809 08:51:15.659773 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:15.659934 10977 external_mini_cluster.cc:1428] Setting key f41e675dfb89156d2aa731a7312b2458
I20260809 08:51:15.660106 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.652231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:32905 (local address 127.10.184.126:37619)
0809 08:51:15.652279 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.652281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.652459 (+ 178us) server_negotiation.cc:407] Connection header received
0809 08:51:15.652919 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.652921 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.652932 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.652990 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.653813 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.654307 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.655191 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.655365 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.655408 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.657250 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.657252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.657252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:15.657260 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.658764 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.659142 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.659151 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.659151 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.659209 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.659459 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.659460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.659460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.659571 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.659626 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.659941 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.660019 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 08:51:15.660255 7151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.652184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:37619 (local address 127.10.184.67:32905)
0809 08:51:15.652343 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.652383 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.652830 (+ 447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.653043 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.653046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.653561 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:15.653749 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.653754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.654364 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.654365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.655134 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.655136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.655415 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.655416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.655511 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.655872 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:15.655879 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:15.657158 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:15.658834 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.658841 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.658843 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.659076 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.659287 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:15.659287 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:15.659288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:15.659402 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:15.659626 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:15.659633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:15.659768 (+ 135us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.659906 (+ 138us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.660056 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":68,"threads_started":1}
I20260809 08:51:15.660844 7148 heartbeater.cc:347] Connected to a master server at 127.10.184.126:37619
I20260809 08:51:15.660955 7148 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:15.661101 7148 heartbeater.cc:511] Master 127.10.184.126:37619 requested a full tablet report, sending...
I20260809 08:51:15.661412 6459 ts_manager.cc:194] Registered new tserver with Master: 96a7ba8836434eda85c95dfb1d2dbfd8 (127.10.184.67:39611)
I20260809 08:51:15.661775 6459 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:32905
I20260809 08:51:15.671449 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:15.678402 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.672698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48898 (local address 127.10.184.126:37619)
0809 08:51:15.672731 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.672733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.672845 (+ 112us) server_negotiation.cc:407] Connection header received
0809 08:51:15.672910 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.672911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.672920 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.672960 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.673486 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.673926 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.674618 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.674797 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.674848 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.676080 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.676082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.676082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:15.676089 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.677341 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.677693 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.677694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.677694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.677746 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:15.677930 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:15.677931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:15.677931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:15.678052 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.678141 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.678264 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.678319 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:51:15.679885 6459 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48898:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:15.681396 6459 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:15.687394 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46026 (local address 127.10.184.67:39611)
0809 08:51:15.683753 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.683757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.683794 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:51:15.683867 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.683868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.683883 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.683952 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:15.684128 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685281 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.686514 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687019 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687080 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687135 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.687283 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.687321 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":77,"threads_started":1}
I20260809 08:51:15.687384 7162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39611 (local address 127.0.0.1:46026)
0809 08:51:15.683741 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.683751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.683805 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.684009 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.684010 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.684012 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:15.684058 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.684059 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685381 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.685382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.686440 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.686441 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687095 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.687186 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687200 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.687261 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.687314 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":4,"thread_start_us":52,"threads_started":1}
I20260809 08:51:15.687520 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39863 (local address 127.0.0.1:33718)
0809 08:51:15.683445 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.683454 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.683510 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.683952 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.683955 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.683957 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:15.684020 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.684038 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685174 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.685175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.686394 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.686397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687307 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.687369 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687373 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.687400 (+ 27us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.687439 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":60,"threads_started":1}
I20260809 08:51:15.687570 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:38365 (local address 127.0.0.1:48274)
0809 08:51:15.683415 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.683450 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.683509 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.684134 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.684135 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.684137 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:15.684170 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.684171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685600 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.685601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.686545 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.686548 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687368 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.687420 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687423 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.687447 (+ 24us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.687478 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"thread_start_us":45,"threads_started":1}
I20260809 08:51:15.687958 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33718 (local address 127.10.184.65:39863)
0809 08:51:15.683803 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.683807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.683817 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:15.683851 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.683853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.683864 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.683931 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:15.684095 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685090 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.686493 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687205 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687636 (+ 431us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687768 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.687834 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.687875 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":89,"threads_started":1}
I20260809 08:51:15.687970 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.683462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48274 (local address 127.10.184.66:38365)
0809 08:51:15.683922 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.683926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.683934 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:15.683973 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.683975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.683989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.684142 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:15.684438 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.685517 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.686624 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.687288 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.687753 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.687813 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.687836 (+ 23us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.687875 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":81,"threads_started":1}
I20260809 08:51:15.688364 7041 tablet_service.cc:1467] Processing CreateTablet for tablet b4957b2563364db986fda1c461b2419d (DEFAULT_TABLE table=test-table [id=407be65e7b1f48fc868cba262fd0b266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:15.688547 6862 tablet_service.cc:1467] Processing CreateTablet for tablet b4957b2563364db986fda1c461b2419d (DEFAULT_TABLE table=test-table [id=407be65e7b1f48fc868cba262fd0b266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:15.688727 7041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4957b2563364db986fda1c461b2419d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:15.688874 6862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4957b2563364db986fda1c461b2419d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:15.691246 7167 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Bootstrap starting.
I20260809 08:51:15.691532 7166 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Bootstrap starting.
I20260809 08:51:15.691515 6664 tablet_service.cc:1467] Processing CreateTablet for tablet b4957b2563364db986fda1c461b2419d (DEFAULT_TABLE table=test-table [id=407be65e7b1f48fc868cba262fd0b266]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:15.691870 6664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4957b2563364db986fda1c461b2419d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:15.691896 7167 tablet_bootstrap.cc:654] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:15.692133 7166 tablet_bootstrap.cc:654] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:15.692373 7167 log.cc:824] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:15.692721 7166 log.cc:824] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:15.693392 7167 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: No bootstrap required, opened a new log
I20260809 08:51:15.693452 7167 ts_tablet_manager.cc:1397] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:15.693516 7166 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: No bootstrap required, opened a new log
I20260809 08:51:15.693586 7166 ts_tablet_manager.cc:1397] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:15.694927 7167 raft_consensus.cc:348] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.695114 7167 raft_consensus.cc:374] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:15.695163 7167 raft_consensus.cc:729] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13a7d578f189441d8d968c490d258512, State: Initialized, Role: FOLLOWER
I20260809 08:51:15.695271 7167 consensus_queue.cc:260] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.695529 7167 ts_tablet_manager.cc:1428] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:15.695569 6952 heartbeater.cc:503] Master 127.10.184.126:37619 was elected leader, sending a full tablet report...
I20260809 08:51:15.695565 7166 raft_consensus.cc:348] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.695713 7166 raft_consensus.cc:374] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:15.695735 7166 raft_consensus.cc:729] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96a7ba8836434eda85c95dfb1d2dbfd8, State: Initialized, Role: FOLLOWER
I20260809 08:51:15.695816 7166 consensus_queue.cc:260] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.696063 7166 ts_tablet_manager.cc:1428] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 08:51:15.696095 7148 heartbeater.cc:503] Master 127.10.184.126:37619 was elected leader, sending a full tablet report...
I20260809 08:51:15.696249 7171 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: Bootstrap starting.
I20260809 08:51:15.696872 7171 tablet_bootstrap.cc:654] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:15.697373 7171 log.cc:824] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:15.697986 7171 tablet_bootstrap.cc:492] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: No bootstrap required, opened a new log
I20260809 08:51:15.698043 7171 ts_tablet_manager.cc:1397] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:15.699447 7171 raft_consensus.cc:348] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.699576 7171 raft_consensus.cc:374] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:15.699607 7171 raft_consensus.cc:729] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 520707c858974d0899e3175ffc8fc99b, State: Initialized, Role: FOLLOWER
I20260809 08:51:15.699712 7171 consensus_queue.cc:260] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.699936 7171 ts_tablet_manager.cc:1428] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:15.700011 6764 heartbeater.cc:503] Master 127.10.184.126:37619 was elected leader, sending a full tablet report...
W20260809 08:51:15.759461 6953 tablet.cc:2369] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:51:15.763198 7178 raft_consensus.cc:482] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:15.763302 7178 raft_consensus.cc:504] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.763670 7178 leader_election.cc:288] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 13a7d578f189441d8d968c490d258512 (127.10.184.66:38365), 96a7ba8836434eda85c95dfb1d2dbfd8 (127.10.184.67:39611)
I20260809 08:51:15.766297 6767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.763817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:38365 (local address 127.10.184.65:39659)
0809 08:51:15.763875 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.763886 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.763941 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.764068 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.764069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.764070 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:15.764139 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.764140 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.764754 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.764755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.765515 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.765518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.766050 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.766051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.766127 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.766131 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.766184 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.766227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 08:51:15.766297 7163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.763877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:39659 (local address 127.10.184.66:38365)
0809 08:51:15.763935 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.763937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.763945 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:15.763995 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.763996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.764008 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.764058 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:15.764201 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.764678 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.765604 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.765975 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.766039 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.766048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.766191 (+ 143us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.766227 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:51:15.766480 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.763954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:39611 (local address 127.10.184.65:47335)
0809 08:51:15.764115 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.764124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:15.764186 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:15.764310 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:15.764311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:15.764312 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:15.764381 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.764383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.765002 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.765003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.765707 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:15.765709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.766247 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.766248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:15.766313 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.766316 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:15.766361 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:51:15.766401 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260809 08:51:15.766571 7164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.764041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:47335 (local address 127.10.184.67:39611)
0809 08:51:15.764105 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.764107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.764183 (+ 76us) server_negotiation.cc:407] Connection header received
0809 08:51:15.764248 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.764249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.764259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.764300 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:15.764441 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.764925 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.765795 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.766172 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.766232 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.766240 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.766468 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.766511 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:51:15.766862 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4957b2563364db986fda1c461b2419d" candidate_uuid: "520707c858974d0899e3175ffc8fc99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a7d578f189441d8d968c490d258512" is_pre_election: true
I20260809 08:51:15.766886 7076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4957b2563364db986fda1c461b2419d" candidate_uuid: "520707c858974d0899e3175ffc8fc99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" is_pre_election: true
I20260809 08:51:15.767001 6888 raft_consensus.cc:2454] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520707c858974d0899e3175ffc8fc99b in term 0.
I20260809 08:51:15.767009 7076 raft_consensus.cc:2454] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 520707c858974d0899e3175ffc8fc99b in term 0.
I20260809 08:51:15.767220 6597 leader_election.cc:302] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 13a7d578f189441d8d968c490d258512, 520707c858974d0899e3175ffc8fc99b; no voters:
I20260809 08:51:15.767357 7178 raft_consensus.cc:2781] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:15.767419 7178 raft_consensus.cc:482] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:15.767436 7178 raft_consensus.cc:3037] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:15.768131 7178 raft_consensus.cc:504] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.768242 7178 leader_election.cc:288] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [CANDIDATE]: Term 1 election: Requested vote from peers 13a7d578f189441d8d968c490d258512 (127.10.184.66:38365), 96a7ba8836434eda85c95dfb1d2dbfd8 (127.10.184.67:39611)
I20260809 08:51:15.768386 6888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4957b2563364db986fda1c461b2419d" candidate_uuid: "520707c858974d0899e3175ffc8fc99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "13a7d578f189441d8d968c490d258512"
I20260809 08:51:15.768419 7076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4957b2563364db986fda1c461b2419d" candidate_uuid: "520707c858974d0899e3175ffc8fc99b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8"
I20260809 08:51:15.768466 6888 raft_consensus.cc:3037] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:15.768479 7076 raft_consensus.cc:3037] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:15.769222 7076 raft_consensus.cc:2454] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520707c858974d0899e3175ffc8fc99b in term 1.
I20260809 08:51:15.769284 6888 raft_consensus.cc:2454] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 520707c858974d0899e3175ffc8fc99b in term 1.
I20260809 08:51:15.769389 6597 leader_election.cc:302] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 520707c858974d0899e3175ffc8fc99b, 96a7ba8836434eda85c95dfb1d2dbfd8; no voters:
I20260809 08:51:15.769476 7178 raft_consensus.cc:2781] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:15.769616 7178 raft_consensus.cc:686] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 1 LEADER]: Becoming Leader. State: Replica: 520707c858974d0899e3175ffc8fc99b, State: Running, Role: LEADER
I20260809 08:51:15.769724 7178 consensus_queue.cc:237] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } }
I20260809 08:51:15.770538 6458 catalog_manager.cc:5495] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b reported cstate change: term changed from 0 to 1, leader changed from <none> to 520707c858974d0899e3175ffc8fc99b (127.10.184.65). New cstate: current_term: 1 leader_uuid: "520707c858974d0899e3175ffc8fc99b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "520707c858974d0899e3175ffc8fc99b" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 39863 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 } health_report { overall_health: UNKNOWN } } }
I20260809 08:51:15.777261 7165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.775305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33720 (local address 127.10.184.65:39863)
0809 08:51:15.775340 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.775341 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.775362 (+ 21us) server_negotiation.cc:407] Connection header received
0809 08:51:15.775418 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.775419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.775434 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.775485 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:15.775575 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.775986 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.776630 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.776794 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.776833 (+ 39us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.776986 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:15.777069 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:15.777132 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.777171 (+ 39us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.777180 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:15.778738 6888 raft_consensus.cc:1261] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 1 FOLLOWER]: Refusing update from remote peer 520707c858974d0899e3175ffc8fc99b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:15.778745 7076 raft_consensus.cc:1261] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 1 FOLLOWER]: Refusing update from remote peer 520707c858974d0899e3175ffc8fc99b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:15.779019 7178 consensus_queue.cc:1035] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "96a7ba8836434eda85c95dfb1d2dbfd8" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 39611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:15.779145 7180 consensus_queue.cc:1035] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [LEADER]: Connected to new peer: Peer: permanent_uuid: "13a7d578f189441d8d968c490d258512" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 38365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:15.781409 7191 mvcc.cc:205] Tried to move back new op lower bound from 7316543388787634176 to 7316543388753383424. Current Snapshot: MvccSnapshot[applied={T|T < 7316543388787634176}]
I20260809 08:51:15.781589 7188 mvcc.cc:205] Tried to move back new op lower bound from 7316543388787634176 to 7316543388753383424. Current Snapshot: MvccSnapshot[applied={T|T < 7316543388787634176}]
I20260809 08:51:15.782464 7189 mvcc.cc:205] Tried to move back new op lower bound from 7316543388787634176 to 7316543388753383424. Current Snapshot: MvccSnapshot[applied={T|T < 7316543388787634176 or (T in {7316543388787634176})}]
I20260809 08:51:15.783946 6458 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:15.784030 6458 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48898:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:15.784698 6458 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67ebe511a1164160b534e181f1c7880d: Sending DeleteTablet for 3 replicas of tablet b4957b2563364db986fda1c461b2419d
I20260809 08:51:15.785054 7041 tablet_service.cc:1514] Processing DeleteTablet for tablet b4957b2563364db986fda1c461b2419d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:15 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:46026
I20260809 08:51:15.785079 6862 tablet_service.cc:1514] Processing DeleteTablet for tablet b4957b2563364db986fda1c461b2419d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:15 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:48274
I20260809 08:51:15.785257 7202 tablet_replica.cc:330] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: stopping tablet replica
I20260809 08:51:15.785257 6664 tablet_service.cc:1514] Processing DeleteTablet for tablet b4957b2563364db986fda1c461b2419d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:15 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:33718
I20260809 08:51:15.785351 7202 raft_consensus.cc:2229] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:15.785445 7202 raft_consensus.cc:2258] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:15.785480 7204 tablet_replica.cc:330] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b: stopping tablet replica
I20260809 08:51:15.785549 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 6573
I20260809 08:51:15.785570 7204 raft_consensus.cc:2229] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:51:15.785662 7204 raft_consensus.cc:2258] T b4957b2563364db986fda1c461b2419d P 520707c858974d0899e3175ffc8fc99b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:15.797281 7202 ts_tablet_manager.cc:1905] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:15.798804 7202 ts_tablet_manager.cc:1918] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:15.798882 7202 log.cc:1196] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/b4957b2563364db986fda1c461b2419d
I20260809 08:51:15.799108 7202 ts_tablet_manager.cc:1939] T b4957b2563364db986fda1c461b2419d P 96a7ba8836434eda85c95dfb1d2dbfd8: Deleting consensus metadata
I20260809 08:51:15.799513 6439 catalog_manager.cc:4841] TS 96a7ba8836434eda85c95dfb1d2dbfd8 (127.10.184.67:39611): tablet b4957b2563364db986fda1c461b2419d (table test-table [id=407be65e7b1f48fc868cba262fd0b266]) successfully deleted
W20260809 08:51:15.800983 6437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:39863 (error 108)
I20260809 08:51:15.801108 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 6771
I20260809 08:51:15.801115 7203 tablet_replica.cc:330] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512: stopping tablet replica
I20260809 08:51:15.801229 7203 raft_consensus.cc:2229] T b4957b2563364db986fda1c461b2419d P 13a7d578f189441d8d968c490d258512 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:15.803901 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.803787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:39863 (local address 127.0.0.1:33728)
0809 08:51:15.803837 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.803873 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:51:15.804015 6437 catalog_manager.cc:4584] TS 520707c858974d0899e3175ffc8fc99b (127.10.184.65:39863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b4957b2563364db986fda1c461b2419d: Network error: Client connection negotiation failed: client connection to 127.10.184.65:39863: connect: Connection refused (error 111)
I20260809 08:51:15.810115 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 6958
I20260809 08:51:15.810226 7160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.810097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:38365 (local address 127.0.0.1:48278)
0809 08:51:15.810152 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:15.810194 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.66:38365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:51:15.816458 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 6418
2026-08-09T08:51:15Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2017 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T08:51:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:15Z Disabled control of system clock
I20260809 08:51:15.851065 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38567
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:38567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:15.929086 7212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:15.929293 7212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:15.929322 7212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:15.930429 7212 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:15.930470 7212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:15.930487 7212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:15.930507 7212 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:15.930523 7212 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:15.931856 7212 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:38567
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:15.932090 7212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:15.932354 7212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:15.937018 7222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:15.937052 7218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.937224 7212 server_base.cc:1034] running on GCE node
W20260809 08:51:15.937057 7219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:15.937481 7212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:15.937750 7212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:15.938889 7212 hybrid_clock.cc:648] HybridClock initialized: now 1786265475938872 us; error 33 us; skew 500 ppm
I20260809 08:51:15.940021 7212 webserver.cc:460] Webserver started at http://127.10.184.126:39553/ using document root <none> and password file <none>
I20260809 08:51:15.940223 7212 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:15.940272 7212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:15.940402 7212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:15.941486 7212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "8ecbec39bde045f48abdd029cbb7de62"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "066647e1053846f406c90f2a013ca248"
server_key_iv: "63c8bfdb4414c4382cbdc177da4f1353"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.941821 7212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "8ecbec39bde045f48abdd029cbb7de62"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "066647e1053846f406c90f2a013ca248"
server_key_iv: "63c8bfdb4414c4382cbdc177da4f1353"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.943033 7212 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:15.943673 7233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:15.943814 7212 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:15.943897 7212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "8ecbec39bde045f48abdd029cbb7de62"
format_stamp: "Formatted at 2026-08-09 08:51:15 on dist-test-slave-ncp9"
server_key: "066647e1053846f406c90f2a013ca248"
server_key_iv: "63c8bfdb4414c4382cbdc177da4f1353"
server_key_version: "encryptionkey@0"
I20260809 08:51:15.943955 7212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:15.962651 7212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:15.962883 7212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:15.962978 7212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:15.966689 7212 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:38567
I20260809 08:51:15.966722 7297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:38567 every 8 connection(s)
I20260809 08:51:15.967010 7212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:15.967926 7298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:15.970135 7298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: Bootstrap starting.
I20260809 08:51:15.970700 7298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:15.970963 7298 log.cc:824] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:15.971560 7298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: No bootstrap required, opened a new log
I20260809 08:51:15.973129 7298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } }
I20260809 08:51:15.973259 7298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:15.973294 7298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ecbec39bde045f48abdd029cbb7de62, State: Initialized, Role: FOLLOWER
I20260809 08:51:15.973426 7298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } }
I20260809 08:51:15.973483 7298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:15.973505 7298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:15.973547 7298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:15.974032 7298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } }
I20260809 08:51:15.974123 7298 leader_election.cc:302] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ecbec39bde045f48abdd029cbb7de62; no voters:
I20260809 08:51:15.974274 7298 leader_election.cc:288] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:15.974319 7305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:15.974471 7305 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [term 1 LEADER]: Becoming Leader. State: Replica: 8ecbec39bde045f48abdd029cbb7de62, State: Running, Role: LEADER
I20260809 08:51:15.974512 7298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:15.974574 7305 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } }
I20260809 08:51:15.974855 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 7212
I20260809 08:51:15.974977 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:15.975206 10977 external_mini_cluster.cc:1428] Setting key 2c4c6dcb2f126cde2ce325002b168862
I20260809 08:51:15.975966 7308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ecbec39bde045f48abdd029cbb7de62. Latest consensus state: current_term: 1 leader_uuid: "8ecbec39bde045f48abdd029cbb7de62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } } }
I20260809 08:51:15.976193 7308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:15.976392 7307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ecbec39bde045f48abdd029cbb7de62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ecbec39bde045f48abdd029cbb7de62" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38567 } } }
I20260809 08:51:15.976464 7307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62 [sys.catalog]: This master's current role is: LEADER
W20260809 08:51:15.976508 7334 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:15.976549 7334 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:15.976603 7335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:15.976800 7335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:15.978272 7335 catalog_manager.cc:1347] Generated new cluster ID: 48fb930e4635450ebd7fdfe8274bb874
I20260809 08:51:15.978323 7335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:15.981082 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:15.975657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.10.184.126:38567)
0809 08:51:15.975862 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:15.975871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:15.975902 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:51:15.976017 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:15.976025 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:15.976092 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:15.976474 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:15.976570 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.977950 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.979026 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:15.979608 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:15.979717 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:15.980393 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:15.980397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:15.980397 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:15.980599 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:15.980638 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:15.980659 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:15.980718 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:15.980794 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 08:51:15.980865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
I20260809 08:51:16.016568 7335 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:16.017366 7335 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:16.041985 7335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: Generated new TSK 0
I20260809 08:51:16.042217 7335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:16.054250 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--builtin_ntp_servers=127.10.184.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:16.135178 7338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:16.135373 7338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:16.135393 7338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:16.136497 7338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:16.136538 7338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:16.136564 7338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:16.137884 7338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:16.138100 7338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:16.138365 7338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:16.143280 7346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:16.143338 7338 server_base.cc:1034] running on GCE node
W20260809 08:51:16.143296 7348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:16.143277 7343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:16.143709 7338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:16.143929 7338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:16.145052 7338 hybrid_clock.cc:648] HybridClock initialized: now 1786265476145043 us; error 26 us; skew 500 ppm
I20260809 08:51:16.148535 7338 webserver.cc:460] Webserver started at http://127.10.184.65:39673/ using document root <none> and password file <none>
I20260809 08:51:16.148759 7338 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:16.148809 7338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:16.148970 7338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:16.150048 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "20e696d3029243aa8af64c0770cbd6f1"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "eabd92ce53cfb87fa7a6620c80606dd8"
server_key_iv: "d57a8561e1ea5dfd6c9852de45f0d582"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.150380 7338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "20e696d3029243aa8af64c0770cbd6f1"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "eabd92ce53cfb87fa7a6620c80606dd8"
server_key_iv: "d57a8561e1ea5dfd6c9852de45f0d582"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.151561 7338 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:16.152176 7356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.152329 7338 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:16.152412 7338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "20e696d3029243aa8af64c0770cbd6f1"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "eabd92ce53cfb87fa7a6620c80606dd8"
server_key_iv: "d57a8561e1ea5dfd6c9852de45f0d582"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.152472 7338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:16.168610 7338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:16.168850 7338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:16.168993 7338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:16.169184 7338 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:16.169479 7338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:16.169512 7338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.169538 7338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:16.169581 7338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.174806 7338 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:46815
I20260809 08:51:16.174865 7514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:46815 every 8 connection(s)
I20260809 08:51:16.175144 7338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:51:16.179839 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 7338
I20260809 08:51:16.179936 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:16.180158 10977 external_mini_cluster.cc:1428] Setting key c097b8e479e592558d8c4826aa4a47f2
I20260809 08:51:16.180178 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.175881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:36245 (local address 127.10.184.126:38567)
0809 08:51:16.175938 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.175940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.176143 (+ 203us) server_negotiation.cc:407] Connection header received
0809 08:51:16.176662 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.176664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.176679 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.176731 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:16.177417 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.177928 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.178877 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.179081 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.179137 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.179898 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:16.179899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:16.179899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:16.179904 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:16.179919 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:16.179923 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:16.179972 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.180085 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.180117 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:51:16.180385 7517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.175848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38567 (local address 127.10.184.65:36245)
0809 08:51:16.176038 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.176069 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.176562 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.176796 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.176800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.176810 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:16.177327 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.177332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.178012 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.178013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.178787 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.178789 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.179157 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.179158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.179286 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.179760 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:16.179761 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:16.179788 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:16.179810 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:16.179987 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:16.179995 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:16.179999 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.180083 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.180190 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260809 08:51:16.180879 7515 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38567
I20260809 08:51:16.180995 7515 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:16.181115 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--builtin_ntp_servers=127.10.184.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 08:51:16.181161 7515 heartbeater.cc:511] Master 127.10.184.126:38567 requested a full tablet report, sending...
I20260809 08:51:16.181684 7249 ts_manager.cc:194] Registered new tserver with Master: 20e696d3029243aa8af64c0770cbd6f1 (127.10.184.65:46815)
I20260809 08:51:16.182492 7249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.65:36245
W20260809 08:51:16.261206 7518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:16.261389 7518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:16.261417 7518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:16.262513 7518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:16.262552 7518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:16.262585 7518 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:16.263881 7518 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:16.264106 7518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:16.264369 7518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:16.269076 7532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:16.269119 7528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:16.269068 7530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:16.269091 7518 server_base.cc:1034] running on GCE node
I20260809 08:51:16.269505 7518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:16.269697 7518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:16.270831 7518 hybrid_clock.cc:648] HybridClock initialized: now 1786265476270814 us; error 33 us; skew 500 ppm
I20260809 08:51:16.272017 7518 webserver.cc:460] Webserver started at http://127.10.184.66:37683/ using document root <none> and password file <none>
I20260809 08:51:16.272244 7518 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:16.272289 7518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:16.272413 7518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:16.273443 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "fe6a3d15894f41f2bb7e835cd664397d"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "7686381c9a951ab1c4a8c6d33560a62c"
server_key_iv: "a36a92485f1ea6d43ae3c9cfe531c0e3"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.273757 7518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "fe6a3d15894f41f2bb7e835cd664397d"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "7686381c9a951ab1c4a8c6d33560a62c"
server_key_iv: "a36a92485f1ea6d43ae3c9cfe531c0e3"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.275036 7518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:16.275713 7539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.275866 7518 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:16.275949 7518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "fe6a3d15894f41f2bb7e835cd664397d"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "7686381c9a951ab1c4a8c6d33560a62c"
server_key_iv: "a36a92485f1ea6d43ae3c9cfe531c0e3"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.276013 7518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:16.284673 7518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:16.284948 7518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:16.285063 7518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:16.285248 7518 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:16.285526 7518 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:16.285558 7518 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.285583 7518 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:16.285609 7518 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.290705 7518 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:46291
I20260809 08:51:16.290768 7714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:46291 every 8 connection(s)
I20260809 08:51:16.291011 7518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:16.295796 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.291606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:54799 (local address 127.10.184.126:38567)
0809 08:51:16.291649 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.291651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.291844 (+ 193us) server_negotiation.cc:407] Connection header received
0809 08:51:16.292343 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.292344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.292358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.292408 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:16.293114 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.293605 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.294553 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.294764 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.294818 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.295543 (+ 725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:16.295545 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:16.295545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:16.295550 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:16.295565 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:16.295570 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:16.295615 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.295686 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.295719 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:16.295969 7717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.291587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38567 (local address 127.10.184.66:54799)
0809 08:51:16.291742 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.291772 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.292244 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.292471 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.292475 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.292485 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:16.293027 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.293033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.293683 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.293684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.294466 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.294468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.294832 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.294833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.294963 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.295412 (+ 449us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:16.295413 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:16.295439 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:16.295461 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:16.295621 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:16.295626 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:16.295629 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.295690 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.295784 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260809 08:51:16.296370 7715 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38567
I20260809 08:51:16.296471 7715 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:16.296456 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 7518
I20260809 08:51:16.296556 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:16.296656 7715 heartbeater.cc:511] Master 127.10.184.126:38567 requested a full tablet report, sending...
I20260809 08:51:16.296767 10977 external_mini_cluster.cc:1428] Setting key 5cac1236b0bf309bee82ecf91f4a8c06
I20260809 08:51:16.297053 7249 ts_manager.cc:194] Registered new tserver with Master: fe6a3d15894f41f2bb7e835cd664397d (127.10.184.66:46291)
I20260809 08:51:16.297468 7249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.66:54799
I20260809 08:51:16.297710 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--builtin_ntp_servers=127.10.184.84:43693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:16.376870 7718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:16.377076 7718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:16.377103 7718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:16.378191 7718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:16.378242 7718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:16.378288 7718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:16.379643 7718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:38567
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:16.379828 7718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:16.380057 7718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:16.384672 7726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:16.384732 7725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:16.384706 7728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:16.384884 7718 server_base.cc:1034] running on GCE node
I20260809 08:51:16.385119 7718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:16.385346 7718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:16.386476 7718 hybrid_clock.cc:648] HybridClock initialized: now 1786265476386468 us; error 26 us; skew 500 ppm
I20260809 08:51:16.387619 7718 webserver.cc:460] Webserver started at http://127.10.184.67:40835/ using document root <none> and password file <none>
I20260809 08:51:16.387847 7718 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:16.387894 7718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:16.388018 7718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:16.389091 7718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "c0c827404f36b34b45df276e789e275c"
server_key_iv: "5a4cbb8ed5edbc3b03109e42b1fa592b"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.389428 7718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "c0c827404f36b34b45df276e789e275c"
server_key_iv: "5a4cbb8ed5edbc3b03109e42b1fa592b"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.390614 7718 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:16.391227 7738 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.391376 7718 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:16.391456 7718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5"
format_stamp: "Formatted at 2026-08-09 08:51:16 on dist-test-slave-ncp9"
server_key: "c0c827404f36b34b45df276e789e275c"
server_key_iv: "5a4cbb8ed5edbc3b03109e42b1fa592b"
server_key_version: "encryptionkey@0"
I20260809 08:51:16.391520 7718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:16.422886 7718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:16.423177 7718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:16.423310 7718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:16.423519 7718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:16.423858 7718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:16.423902 7718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.423933 7718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:16.423949 7718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:16.429296 7718 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:37009
I20260809 08:51:16.429337 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:37009 every 8 connection(s)
I20260809 08:51:16.429644 7718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:51:16.432677 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 7718
I20260809 08:51:16.432762 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:16.432943 10977 external_mini_cluster.cc:1428] Setting key eae20d6a651c99616ff50d4452b40d76
I20260809 08:51:16.434798 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.430336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:46205 (local address 127.10.184.126:38567)
0809 08:51:16.430384 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.430386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.430615 (+ 229us) server_negotiation.cc:407] Connection header received
0809 08:51:16.431082 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.431083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.431092 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.431156 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:16.431801 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.432295 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.433427 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.433694 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.433755 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.434529 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:16.434530 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:16.434531 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:16.434535 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:16.434551 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:16.434555 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:16.434602 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.434687 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.434718 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:16.434976 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.430318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:38567 (local address 127.10.184.67:46205)
0809 08:51:16.430500 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.430540 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.431007 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.431213 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.431216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.431224 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:16.431734 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.431739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.432355 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.432356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.433350 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.433354 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.433774 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.433775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.433903 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.434395 (+ 492us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:16.434395 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:16.434424 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:16.434446 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:16.434608 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:16.434612 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:16.434615 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.434684 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.434786 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260809 08:51:16.435387 7893 heartbeater.cc:347] Connected to a master server at 127.10.184.126:38567
I20260809 08:51:16.435459 7893 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:16.435599 7893 heartbeater.cc:511] Master 127.10.184.126:38567 requested a full tablet report, sending...
I20260809 08:51:16.435897 7249 ts_manager.cc:194] Registered new tserver with Master: 7a7f1ecf8e334fb98f2b51fbb2a51ee5 (127.10.184.67:37009)
I20260809 08:51:16.436266 7249 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.67:46205
I20260809 08:51:16.445492 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:16.449501 7324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.446977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.10.184.126:38567)
0809 08:51:16.447023 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.447025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.447155 (+ 130us) server_negotiation.cc:407] Connection header received
0809 08:51:16.447216 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.447217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.447229 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.447265 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:16.447387 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.447875 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.448535 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.448706 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.448746 (+ 40us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.449298 (+ 552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:16.449300 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:16.449300 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:16.449304 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:16.449317 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:16.449321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:16.449382 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.449423 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.449432 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:16.451076 7249 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52714:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:16.452555 7249 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:16.458479 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:46291 (local address 127.0.0.1:60022)
0809 08:51:16.454936 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.454944 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.454981 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.455388 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.455389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.455392 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:16.455428 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.455429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456685 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.456686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.457573 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.457576 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458199 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.458283 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458298 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.458357 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.458407 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":65,"threads_started":1}
I20260809 08:51:16.458493 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.10.184.66:46291)
0809 08:51:16.455108 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.455111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.455118 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:16.455181 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.455183 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.455199 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.455266 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:16.455507 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456610 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.457645 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458123 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458196 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458288 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.458383 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.458419 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":96,"threads_started":1}
I20260809 08:51:16.458827 7907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:37009 (local address 127.0.0.1:47320)
0809 08:51:16.454642 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.454691 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.454756 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.455305 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.455306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.455308 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:16.455350 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.455352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456287 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.456288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.457570 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.457573 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458562 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.458629 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458633 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.458690 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.458731 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":2,"thread_start_us":67,"threads_started":1}
I20260809 08:51:16.458843 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47320 (local address 127.10.184.67:37009)
0809 08:51:16.455081 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.455086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.455095 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:16.455130 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.455132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.455145 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.455243 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:16.455456 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456215 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.457669 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458378 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458445 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458523 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.458722 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.458764 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":83,"threads_started":1}
I20260809 08:51:16.458988 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53384 (local address 127.10.184.65:46815)
0809 08:51:16.454839 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.454844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.454919 (+ 75us) server_negotiation.cc:407] Connection header received
0809 08:51:16.454970 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.454972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.454989 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.455066 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:16.455306 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456412 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.457678 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458378 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458445 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458524 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.458870 (+ 346us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.458912 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":104,"threads_started":1}
I20260809 08:51:16.458827 7906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.454527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:46815 (local address 127.0.0.1:53384)
0809 08:51:16.454837 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.454848 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.454911 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.455145 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.455148 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.455150 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:16.455222 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.455224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.456490 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.456491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.457574 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.457576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.458561 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.458562 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.458629 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.458633 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.458690 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.458731 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":7,"thread_start_us":52,"threads_started":1}
I20260809 08:51:16.459694 7804 tablet_service.cc:1467] Processing CreateTablet for tablet af3bb540305a410c9bdfc2e7b055433d (DEFAULT_TABLE table=test-table [id=e614401846e647f2bfbeb994fa652808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:16.459707 7621 tablet_service.cc:1467] Processing CreateTablet for tablet af3bb540305a410c9bdfc2e7b055433d (DEFAULT_TABLE table=test-table [id=e614401846e647f2bfbeb994fa652808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:16.460019 7804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3bb540305a410c9bdfc2e7b055433d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:16.460049 7621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3bb540305a410c9bdfc2e7b055433d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:16.459975 7415 tablet_service.cc:1467] Processing CreateTablet for tablet af3bb540305a410c9bdfc2e7b055433d (DEFAULT_TABLE table=test-table [id=e614401846e647f2bfbeb994fa652808]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:16.460299 7415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet af3bb540305a410c9bdfc2e7b055433d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:16.462720 7914 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Bootstrap starting.
I20260809 08:51:16.463274 7915 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Bootstrap starting.
I20260809 08:51:16.463393 7914 tablet_bootstrap.cc:654] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:16.463799 7914 log.cc:824] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:16.463954 7915 tablet_bootstrap.cc:654] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:16.464184 7917 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Bootstrap starting.
I20260809 08:51:16.464344 7915 log.cc:824] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:16.464530 7914 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: No bootstrap required, opened a new log
I20260809 08:51:16.464600 7914 ts_tablet_manager.cc:1397] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:16.464776 7917 tablet_bootstrap.cc:654] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:16.465082 7917 log.cc:824] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:16.465178 7915 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: No bootstrap required, opened a new log
I20260809 08:51:16.465241 7915 ts_tablet_manager.cc:1397] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:16.465857 7917 tablet_bootstrap.cc:492] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: No bootstrap required, opened a new log
I20260809 08:51:16.465924 7917 ts_tablet_manager.cc:1397] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:51:16.467144 7915 raft_consensus.cc:348] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.467111 7914 raft_consensus.cc:348] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.467289 7915 raft_consensus.cc:374] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:16.467301 7914 raft_consensus.cc:374] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:16.467329 7915 raft_consensus.cc:729] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe6a3d15894f41f2bb7e835cd664397d, State: Initialized, Role: FOLLOWER
I20260809 08:51:16.467339 7914 raft_consensus.cc:729] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a7f1ecf8e334fb98f2b51fbb2a51ee5, State: Initialized, Role: FOLLOWER
I20260809 08:51:16.467439 7914 consensus_queue.cc:260] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.467434 7915 consensus_queue.cc:260] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.467691 7914 ts_tablet_manager.cc:1428] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:51:16.467702 7915 ts_tablet_manager.cc:1428] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 08:51:16.467821 7715 heartbeater.cc:503] Master 127.10.184.126:38567 was elected leader, sending a full tablet report...
I20260809 08:51:16.467826 7893 heartbeater.cc:503] Master 127.10.184.126:38567 was elected leader, sending a full tablet report...
I20260809 08:51:16.468261 7917 raft_consensus.cc:348] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.468397 7917 raft_consensus.cc:374] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:16.468429 7917 raft_consensus.cc:729] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20e696d3029243aa8af64c0770cbd6f1, State: Initialized, Role: FOLLOWER
I20260809 08:51:16.468519 7917 consensus_queue.cc:260] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.468762 7917 ts_tablet_manager.cc:1428] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:51:16.469094 7515 heartbeater.cc:503] Master 127.10.184.126:38567 was elected leader, sending a full tablet report...
W20260809 08:51:16.541517 7716 tablet.cc:2369] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:51:16.675820 7516 tablet.cc:2369] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 08:51:16.680148 7894 tablet.cc:2369] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:51:16.733522 7922 raft_consensus.cc:482] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:16.733672 7922 raft_consensus.cc:504] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.734046 7922 leader_election.cc:288] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20e696d3029243aa8af64c0770cbd6f1 (127.10.184.65:46815), fe6a3d15894f41f2bb7e835cd664397d (127.10.184.66:46291)
I20260809 08:51:16.737015 7909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.734347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:42077 (local address 127.10.184.65:46815)
0809 08:51:16.734403 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.734405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.734420 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:51:16.734496 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.734498 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.734512 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.734593 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:16.734759 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.735232 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736326 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.736673 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736733 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.736744 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.736881 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.736939 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:51:16.737015 7895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.734268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:46815 (local address 127.10.184.67:42077)
0809 08:51:16.734327 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.734340 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.734392 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.734611 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.734613 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.734615 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:16.734686 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.734688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.735325 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.735327 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.736247 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.736250 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.736750 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.736824 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.736827 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.736870 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.736948 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:51:16.737175 7913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.734473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48675 (local address 127.10.184.66:46291)
0809 08:51:16.734512 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.734514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.734615 (+ 101us) server_negotiation.cc:407] Connection header received
0809 08:51:16.734671 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.734672 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.734682 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.734734 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:16.734878 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.735387 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736437 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.736808 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736880 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.736930 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.737070 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.737106 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:16.737187 7927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.734380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:46291 (local address 127.10.184.67:48675)
0809 08:51:16.734543 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.734553 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:16.734604 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:16.734742 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:16.734743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:16.734744 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:16.734815 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.734817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.735480 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.735482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.736345 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:16.736349 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.736914 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.736915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:16.737006 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.737009 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:16.737063 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:51:16.737102 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":1,"thread_start_us":115,"threads_started":1}
I20260809 08:51:16.737417 7447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3bb540305a410c9bdfc2e7b055433d" candidate_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e696d3029243aa8af64c0770cbd6f1" is_pre_election: true
I20260809 08:51:16.737586 7447 raft_consensus.cc:2454] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a7f1ecf8e334fb98f2b51fbb2a51ee5 in term 0.
I20260809 08:51:16.737571 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3bb540305a410c9bdfc2e7b055433d" candidate_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a3d15894f41f2bb7e835cd664397d" is_pre_election: true
I20260809 08:51:16.737722 7648 raft_consensus.cc:2454] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a7f1ecf8e334fb98f2b51fbb2a51ee5 in term 0.
I20260809 08:51:16.737834 7741 leader_election.cc:302] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20e696d3029243aa8af64c0770cbd6f1, 7a7f1ecf8e334fb98f2b51fbb2a51ee5; no voters:
I20260809 08:51:16.737962 7922 raft_consensus.cc:2781] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:16.738023 7922 raft_consensus.cc:482] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:16.738039 7922 raft_consensus.cc:3037] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:16.738715 7922 raft_consensus.cc:504] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.738822 7922 leader_election.cc:288] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [CANDIDATE]: Term 1 election: Requested vote from peers 20e696d3029243aa8af64c0770cbd6f1 (127.10.184.65:46815), fe6a3d15894f41f2bb7e835cd664397d (127.10.184.66:46291)
I20260809 08:51:16.738997 7648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3bb540305a410c9bdfc2e7b055433d" candidate_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fe6a3d15894f41f2bb7e835cd664397d"
I20260809 08:51:16.739006 7447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "af3bb540305a410c9bdfc2e7b055433d" candidate_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20e696d3029243aa8af64c0770cbd6f1"
I20260809 08:51:16.739069 7648 raft_consensus.cc:3037] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:16.739080 7447 raft_consensus.cc:3037] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:16.739786 7648 raft_consensus.cc:2454] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a7f1ecf8e334fb98f2b51fbb2a51ee5 in term 1.
I20260809 08:51:16.739843 7447 raft_consensus.cc:2454] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a7f1ecf8e334fb98f2b51fbb2a51ee5 in term 1.
I20260809 08:51:16.739965 7741 leader_election.cc:302] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7a7f1ecf8e334fb98f2b51fbb2a51ee5, fe6a3d15894f41f2bb7e835cd664397d; no voters:
I20260809 08:51:16.740051 7922 raft_consensus.cc:2781] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:16.740208 7922 raft_consensus.cc:686] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 1 LEADER]: Becoming Leader. State: Replica: 7a7f1ecf8e334fb98f2b51fbb2a51ee5, State: Running, Role: LEADER
I20260809 08:51:16.740300 7922 consensus_queue.cc:237] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } }
I20260809 08:51:16.741183 7250 catalog_manager.cc:5495] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a7f1ecf8e334fb98f2b51fbb2a51ee5 (127.10.184.67). New cstate: current_term: 1 leader_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a7f1ecf8e334fb98f2b51fbb2a51ee5" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 37009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 } health_report { overall_health: UNKNOWN } } }
I20260809 08:51:16.810024 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.807662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.10.184.67:37009)
0809 08:51:16.807720 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:16.807722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:16.807731 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:16.807812 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:16.807813 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:16.807833 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:16.807891 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:16.808016 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.808488 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.809248 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:16.809433 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:16.809490 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:16.809655 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:16.809745 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:16.809829 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:16.809924 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 08:51:16.809932 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 08:51:16.811738 7648 raft_consensus.cc:1261] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 1 FOLLOWER]: Refusing update from remote peer 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:16.811761 7447 raft_consensus.cc:1261] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 1 FOLLOWER]: Refusing update from remote peer 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:16.812076 7922 consensus_queue.cc:1035] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20e696d3029243aa8af64c0770cbd6f1" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 46815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:16.812222 7929 consensus_queue.cc:1035] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fe6a3d15894f41f2bb7e835cd664397d" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 46291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:16.814379 7935 mvcc.cc:205] Tried to move back new op lower bound from 7316543393018433536 to 7316543392728866816. Current Snapshot: MvccSnapshot[applied={T|T < 7316543393018433536}]
I20260809 08:51:16.814620 7936 mvcc.cc:205] Tried to move back new op lower bound from 7316543393018433536 to 7316543392728866816. Current Snapshot: MvccSnapshot[applied={T|T < 7316543393018433536}]
I20260809 08:51:16.815228 7938 mvcc.cc:205] Tried to move back new op lower bound from 7316543393018433536 to 7316543392728866816. Current Snapshot: MvccSnapshot[applied={T|T < 7316543393018433536 or (T in {7316543393018433536})}]
I20260809 08:51:16.816567 7249 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:16.816661 7249 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52714:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:16.817384 7249 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ecbec39bde045f48abdd029cbb7de62: Sending DeleteTablet for 3 replicas of tablet af3bb540305a410c9bdfc2e7b055433d
I20260809 08:51:16.817703 7804 tablet_service.cc:1514] Processing DeleteTablet for tablet af3bb540305a410c9bdfc2e7b055433d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:16 UTC) from {username='slave'} at 127.0.0.1:47320
I20260809 08:51:16.817782 7415 tablet_service.cc:1514] Processing DeleteTablet for tablet af3bb540305a410c9bdfc2e7b055433d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:16 UTC) from {username='slave'} at 127.0.0.1:53384
I20260809 08:51:16.817988 7949 tablet_replica.cc:330] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: stopping tablet replica
I20260809 08:51:16.818045 7950 tablet_replica.cc:330] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: stopping tablet replica
I20260809 08:51:16.818094 7621 tablet_service.cc:1514] Processing DeleteTablet for tablet af3bb540305a410c9bdfc2e7b055433d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:16 UTC) from {username='slave'} at 127.0.0.1:60022
I20260809 08:51:16.818127 7949 raft_consensus.cc:2229] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:51:16.818161 7950 raft_consensus.cc:2229] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:16.818250 7949 raft_consensus.cc:2258] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:16.818262 7950 raft_consensus.cc:2258] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:16.818485 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 7338
I20260809 08:51:16.818681 7949 ts_tablet_manager.cc:1905] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:16.818696 7950 ts_tablet_manager.cc:1905] T af3bb540305a410c9bdfc2e7b055433d P 20e696d3029243aa8af64c0770cbd6f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:16.819007 7951 tablet_replica.cc:330] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: stopping tablet replica
I20260809 08:51:16.819096 7951 raft_consensus.cc:2229] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:16.819185 7951 raft_consensus.cc:2258] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:16.820600 7949 ts_tablet_manager.cc:1918] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:16.820657 7949 log.cc:1196] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/af3bb540305a410c9bdfc2e7b055433d
I20260809 08:51:16.820858 7949 ts_tablet_manager.cc:1939] T af3bb540305a410c9bdfc2e7b055433d P 7a7f1ecf8e334fb98f2b51fbb2a51ee5: Deleting consensus metadata
I20260809 08:51:16.821276 7235 catalog_manager.cc:4841] TS 7a7f1ecf8e334fb98f2b51fbb2a51ee5 (127.10.184.67:37009): tablet af3bb540305a410c9bdfc2e7b055433d (table test-table [id=e614401846e647f2bfbeb994fa652808]) successfully deleted
I20260809 08:51:16.823139 7951 ts_tablet_manager.cc:1905] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:16.824344 7951 ts_tablet_manager.cc:1918] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:16.824398 7951 log.cc:1196] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/af3bb540305a410c9bdfc2e7b055433d
I20260809 08:51:16.824589 7951 ts_tablet_manager.cc:1939] T af3bb540305a410c9bdfc2e7b055433d P fe6a3d15894f41f2bb7e835cd664397d: Deleting consensus metadata
I20260809 08:51:16.824952 7236 catalog_manager.cc:4841] TS fe6a3d15894f41f2bb7e835cd664397d (127.10.184.66:46291): tablet af3bb540305a410c9bdfc2e7b055433d (table test-table [id=e614401846e647f2bfbeb994fa652808]) successfully deleted
W20260809 08:51:16.834939 7236 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:46815 (error 108)
I20260809 08:51:16.835219 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 7518
I20260809 08:51:16.836644 7911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:16.835330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:46815 (local address 127.0.0.1:53394)
0809 08:51:16.836538 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:16.836602 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:46815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1181}
W20260809 08:51:16.842290 7236 catalog_manager.cc:4584] TS 20e696d3029243aa8af64c0770cbd6f1 (127.10.184.65:46815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet af3bb540305a410c9bdfc2e7b055433d: Network error: Client connection negotiation failed: client connection to 127.10.184.65:46815: connect: Connection refused (error 111)
I20260809 08:51:16.850049 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 7718
I20260809 08:51:16.859843 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 7212
2026-08-09T08:51:16Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1036 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:16 dist-test-slave-ncp9 krb5kdc[7957](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:16 dist-test-slave-ncp9 krb5kdc[7957](info): set up 2 sockets
Aug 09 08:51:16 dist-test-slave-ncp9 krb5kdc[7957](info): commencing operation
krb5kdc: starting...
W20260809 08:51:17.980286 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:18.088207 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41491
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:41491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:18.167281 7973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:18.167470 7973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:18.167490 7973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:18.168555 7973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:18.168587 7973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:18.168599 7973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:18.168612 7973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:18.168627 7973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:18.170058 7973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:41491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:41491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:18.170240 7973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:18.170409 7973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:18.174969 7988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.175014 7984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.175239 7973 server_base.cc:1034] running on GCE node
W20260809 08:51:18.175014 7986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.175532 7973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:18.175806 7973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:18.176945 7973 hybrid_clock.cc:648] HybridClock initialized: now 1786265478176924 us; error 31 us; skew 500 ppm
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:18.179280 7973 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:18.179661 7973 webserver.cc:460] Webserver started at http://127.10.184.126:35085/ using document root <none> and password file <none>
I20260809 08:51:18.179880 7973 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:18.179929 7973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:18.180049 7973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:18.181061 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "da321fe0ca104aaeaac8f8450363e68c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "04311ebdaf767f5900cc032d47bb5784"
server_key_iv: "c285fd9041d20e1c2d52b5c31d30298e"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.181474 7973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "da321fe0ca104aaeaac8f8450363e68c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "04311ebdaf767f5900cc032d47bb5784"
server_key_iv: "c285fd9041d20e1c2d52b5c31d30298e"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.182776 7973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:18.183394 7997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.183542 7973 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:18.183625 7973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "da321fe0ca104aaeaac8f8450363e68c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "04311ebdaf767f5900cc032d47bb5784"
server_key_iv: "c285fd9041d20e1c2d52b5c31d30298e"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.183694 7973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:18.202389 7973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:18.204375 7973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:18.204509 7973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:18.208179 7973 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:41491
I20260809 08:51:18.208212 8077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:41491 every 8 connection(s)
I20260809 08:51:18.208503 7973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:18.209354 8079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:18.211472 8079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: Bootstrap starting.
I20260809 08:51:18.211992 8079 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:18.212258 8079 log.cc:824] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:18.212358 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 7973
I20260809 08:51:18.212481 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:18.212702 10977 external_mini_cluster.cc:1428] Setting key 2e1b3497855c55732ae629076d917dae
I20260809 08:51:18.212996 8079 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: No bootstrap required, opened a new log
I20260809 08:51:18.215059 8079 raft_consensus.cc:348] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } }
I20260809 08:51:18.215209 8079 raft_consensus.cc:374] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:18.215252 8079 raft_consensus.cc:729] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da321fe0ca104aaeaac8f8450363e68c, State: Initialized, Role: FOLLOWER
I20260809 08:51:18.215375 8079 consensus_queue.cc:260] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } }
I20260809 08:51:18.215446 8079 raft_consensus.cc:388] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:18.215476 8079 raft_consensus.cc:482] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:18.215521 8079 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:18.216022 8079 raft_consensus.cc:504] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } }
I20260809 08:51:18.216117 8079 leader_election.cc:302] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da321fe0ca104aaeaac8f8450363e68c; no voters:
I20260809 08:51:18.216284 8079 leader_election.cc:288] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:18.216321 8085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:18.216465 8085 raft_consensus.cc:686] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [term 1 LEADER]: Becoming Leader. State: Replica: da321fe0ca104aaeaac8f8450363e68c, State: Running, Role: LEADER
I20260809 08:51:18.216523 8079 sys_catalog.cc:564] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:18.216620 8085 consensus_queue.cc:237] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } }
I20260809 08:51:18.217051 8086 sys_catalog.cc:455] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da321fe0ca104aaeaac8f8450363e68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } } }
I20260809 08:51:18.217124 8086 sys_catalog.cc:458] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:18.217079 8087 sys_catalog.cc:455] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da321fe0ca104aaeaac8f8450363e68c. Latest consensus state: current_term: 1 leader_uuid: "da321fe0ca104aaeaac8f8450363e68c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da321fe0ca104aaeaac8f8450363e68c" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 41491 } } }
I20260809 08:51:18.217162 8087 sys_catalog.cc:458] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:18.217423 8094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:18.217903 8094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:18.220469 8094 catalog_manager.cc:1347] Generated new cluster ID: 95d153b4c00a4e54a2082641a7df1d61
I20260809 08:51:18.220521 8094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:18.224519 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.213179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46978 (local address 127.10.184.126:41491)
0809 08:51:18.213343 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.213351 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.213374 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:51:18.214153 (+ 779us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.214159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.214215 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.214324 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.215055 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.216286 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.217354 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.217941 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.218043 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.220657 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.220662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.220662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:18.220673 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.222568 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.222944 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.222947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.222948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.223068 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.223241 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.223241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.223242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.223537 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.224099 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.224185 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.224326 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260809 08:51:18.227849 8094 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:18.228315 8094 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:18.240373 8094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: Generated new TSK 0
I20260809 08:51:18.240530 8094 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.10.184.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
Entry for principal kudu/127.10.184.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab.
I20260809 08:51:18.266981 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:18.345675 8112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:18.345860 8112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:18.345881 8112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:18.346982 8112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:18.347023 8112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:18.347051 8112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:18.348444 8112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.65.keytab
--principal=kudu/127.10.184.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:18.348652 8112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:18.348853 8112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:18.353602 8119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.353610 8124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.353618 8120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.353763 8112 server_base.cc:1034] running on GCE node
I20260809 08:51:18.354015 8112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:18.354218 8112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:18.355350 8112 hybrid_clock.cc:648] HybridClock initialized: now 1786265478355339 us; error 28 us; skew 500 ppm
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:18.357661 8112 init.cc:377] Logged in from keytab as kudu/127.10.184.65@KRBTEST.COM (short username kudu)
I20260809 08:51:18.358042 8112 webserver.cc:460] Webserver started at http://127.10.184.65:44091/ using document root <none> and password file <none>
I20260809 08:51:18.358232 8112 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:18.358279 8112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:18.358402 8112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:18.359364 8112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "1363ec5b0cca41faaefe5dd256da0f4c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "27598be3f326f0a99c1ba0154e3d3286"
server_key_iv: "2c4de309851ee4a766b08b1e727f9c47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.359690 8112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "1363ec5b0cca41faaefe5dd256da0f4c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "27598be3f326f0a99c1ba0154e3d3286"
server_key_iv: "2c4de309851ee4a766b08b1e727f9c47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.360924 8112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:18.361590 8134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.361750 8112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:18.361842 8112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "1363ec5b0cca41faaefe5dd256da0f4c"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "27598be3f326f0a99c1ba0154e3d3286"
server_key_iv: "2c4de309851ee4a766b08b1e727f9c47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.361909 8112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:18.374361 8112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:18.375881 8112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:18.376003 8112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:18.376199 8112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:18.376457 8112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:18.376490 8112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.376518 8112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:18.376541 8112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.382014 8112 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:35745
I20260809 08:51:18.382048 8304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:35745 every 8 connection(s)
I20260809 08:51:18.382337 8112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.65@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:18.390250 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.382974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:60077 (local address 127.10.184.126:41491)
0809 08:51:18.383025 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.383027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.383194 (+ 167us) server_negotiation.cc:407] Connection header received
0809 08:51:18.383653 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.383655 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.383671 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.383720 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.384537 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.385090 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.386011 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.386212 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.386249 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.388098 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.388100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.388100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:18.388107 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.389239 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.389563 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.389563 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.389564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.389612 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.389784 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.389784 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.389784 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.389879 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.389924 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.390114 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.390185 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:51:18.390407 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.382937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41491 (local address 127.10.184.65:60077)
0809 08:51:18.383079 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.383117 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.383552 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.383771 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.383774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.384296 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:18.384472 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.384476 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.385159 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.385160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.385953 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.385956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.386266 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.386267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.386363 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.386732 (+ 369us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:18.386740 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:18.388019 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:18.389302 (+ 1283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.389308 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.389310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.389507 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.389657 (+ 150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.389658 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.389658 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.389743 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.389920 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:18.389923 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:18.390025 (+ 102us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.390097 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.390217 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
I20260809 08:51:18.390874 8305 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41491
I20260809 08:51:18.390959 8305 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:18.391125 8305 heartbeater.cc:511] Master 127.10.184.126:41491 requested a full tablet report, sending...
I20260809 08:51:18.391260 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 8112
I20260809 08:51:18.391377 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:18.391571 10977 external_mini_cluster.cc:1428] Setting key 0d73a1c9d90cda83b6318a3f641718ac
I20260809 08:51:18.391676 8022 ts_manager.cc:194] Registered new tserver with Master: 1363ec5b0cca41faaefe5dd256da0f4c (127.10.184.65:35745)
I20260809 08:51:18.392395 8022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.65@KRBTEST.COM'} at 127.10.184.65:60077
WARNING: no policy specified for kudu/127.10.184.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
Entry for principal kudu/127.10.184.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab.
I20260809 08:51:18.413268 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:18.494889 8313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:18.495080 8313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:18.495111 8313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:18.496207 8313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:18.496248 8313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:18.496280 8313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:18.497736 8313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.66.keytab
--principal=kudu/127.10.184.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:18.497969 8313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:18.498169 8313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:18.502811 8322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.502846 8324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.502948 8313 server_base.cc:1034] running on GCE node
W20260809 08:51:18.502840 8321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.503221 8313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:18.503460 8313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:18.504601 8313 hybrid_clock.cc:648] HybridClock initialized: now 1786265478504577 us; error 36 us; skew 500 ppm
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:18.506928 8313 init.cc:377] Logged in from keytab as kudu/127.10.184.66@KRBTEST.COM (short username kudu)
I20260809 08:51:18.507334 8313 webserver.cc:460] Webserver started at http://127.10.184.66:37647/ using document root <none> and password file <none>
I20260809 08:51:18.507532 8313 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:18.507579 8313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:18.507701 8313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:18.508638 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "33a4df817ff940bcbe9473753e9184b4"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "af5993ebbef1ed755c13b67f761f6c23"
server_key_iv: "1f8366ea838a4462a7f8af9eec567d8b"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.509004 8313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "33a4df817ff940bcbe9473753e9184b4"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "af5993ebbef1ed755c13b67f761f6c23"
server_key_iv: "1f8366ea838a4462a7f8af9eec567d8b"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.510270 8313 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:18.510898 8332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.511035 8313 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.511116 8313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "33a4df817ff940bcbe9473753e9184b4"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "af5993ebbef1ed755c13b67f761f6c23"
server_key_iv: "1f8366ea838a4462a7f8af9eec567d8b"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.511185 8313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:18.525689 8313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:18.527628 8313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:18.527761 8313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:18.527953 8313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:18.528226 8313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:18.528259 8313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.528286 8313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:18.528304 8313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.533514 8313 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:43639
I20260809 08:51:18.533578 8495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:43639 every 8 connection(s)
I20260809 08:51:18.533838 8313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:18.537865 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 8313
I20260809 08:51:18.537982 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:18.538163 10977 external_mini_cluster.cc:1428] Setting key 8573b9c194dbc75f76399c555c354609
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.66@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:18.542220 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.534461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:58551 (local address 127.10.184.126:41491)
0809 08:51:18.534505 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.534508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.534703 (+ 195us) server_negotiation.cc:407] Connection header received
0809 08:51:18.535125 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.535125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.535136 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.535189 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.536016 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.536501 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.537443 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.537619 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.537668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.539692 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.539694 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.539694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:18.539702 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.540970 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.541337 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.541338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.541339 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.541396 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.541608 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.541609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.541609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.541731 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.541787 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.542035 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.542115 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:18.542382 8501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.534422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41491 (local address 127.10.184.66:58551)
0809 08:51:18.534576 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.534615 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.535051 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.535241 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.535243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.535774 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:18.535951 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.535955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.536560 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.536561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.537379 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.537382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.537676 (+ 294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.537676 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.537766 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.538251 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:18.538260 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:18.539594 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:18.541045 (+ 1451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.541052 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.541054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.541270 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.541448 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.541449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.541449 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.541553 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.541786 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:18.541792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:18.541919 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.542003 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.542152 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":83,"threads_started":1}
I20260809 08:51:18.542938 8496 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41491
I20260809 08:51:18.543030 8496 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:18.543190 8496 heartbeater.cc:511] Master 127.10.184.126:41491 requested a full tablet report, sending...
I20260809 08:51:18.543561 8022 ts_manager.cc:194] Registered new tserver with Master: 33a4df817ff940bcbe9473753e9184b4 (127.10.184.66:43639)
I20260809 08:51:18.543916 8022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.66@KRBTEST.COM'} at 127.10.184.66:58551
WARNING: no policy specified for kudu/127.10.184.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
Entry for principal kudu/127.10.184.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab.
I20260809 08:51:18.561753 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:18.640617 8504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:18.640802 8504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:18.640825 8504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:18.641918 8504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:18.641958 8504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:18.641983 8504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:18.643360 8504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.67.keytab
--principal=kudu/127.10.184.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:41491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:18.643527 8504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:18.643699 8504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:18.648300 8514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.648306 8515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.648409 8504 server_base.cc:1034] running on GCE node
W20260809 08:51:18.648306 8517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.648677 8504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:18.648916 8504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:18.650055 8504 hybrid_clock.cc:648] HybridClock initialized: now 1786265478650046 us; error 28 us; skew 500 ppm
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:18.652361 8504 init.cc:377] Logged in from keytab as kudu/127.10.184.67@KRBTEST.COM (short username kudu)
I20260809 08:51:18.652783 8504 webserver.cc:460] Webserver started at http://127.10.184.67:33631/ using document root <none> and password file <none>
I20260809 08:51:18.653023 8504 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:18.653067 8504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:18.653194 8504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:18.654227 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "c451fe718da04637879e376572867c7f"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "5077ff792edd78f735ce58d65d328f40"
server_key_iv: "4e655742c8a3e419336dd2b552c0ce04"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.654548 8504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "c451fe718da04637879e376572867c7f"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "5077ff792edd78f735ce58d65d328f40"
server_key_iv: "4e655742c8a3e419336dd2b552c0ce04"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.655728 8504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:18.656351 8524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.656498 8504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:18.656577 8504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "c451fe718da04637879e376572867c7f"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "5077ff792edd78f735ce58d65d328f40"
server_key_iv: "4e655742c8a3e419336dd2b552c0ce04"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.656641 8504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:18.685130 8504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:18.686875 8504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:18.686997 8504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:18.687203 8504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:18.687475 8504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:18.687507 8504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.687528 8504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:18.687551 8504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.692730 8504 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:45673
I20260809 08:51:18.692785 8676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:45673 every 8 connection(s)
I20260809 08:51:18.693084 8504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:51:18.696346 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 8504
I20260809 08:51:18.696444 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:18.696602 10977 external_mini_cluster.cc:1428] Setting key 7a5dd55304f752dd1fe472fc7718a56a
Aug 09 08:51:18 dist-test-slave-ncp9 krb5kdc[7957](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265478, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.67@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:18.701506 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.693695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:33301 (local address 127.10.184.126:41491)
0809 08:51:18.693755 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.693757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.693943 (+ 186us) server_negotiation.cc:407] Connection header received
0809 08:51:18.694371 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.694372 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.694382 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.694430 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.695411 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.695906 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.697054 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.697331 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.697398 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.699326 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.699328 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.699328 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:18.699335 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.700446 (+ 1111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.700780 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.700780 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.700781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.700828 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.701036 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.701036 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.701037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.701136 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.701188 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.701383 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.701442 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:51:18.701680 8682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.693681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:41491 (local address 127.10.184.67:33301)
0809 08:51:18.693838 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.693872 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.694299 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.694492 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.694495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.695035 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:18.695326 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.695333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.695986 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.695987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.696989 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.696992 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.697409 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.697410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.697584 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.697979 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:18.697987 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:18.699249 (+ 1262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:18.700514 (+ 1265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.700519 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.700521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.700726 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.700871 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:18.700872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:18.700872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:18.700989 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:18.701184 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:18.701187 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:18.701294 (+ 107us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.701368 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.701484 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
I20260809 08:51:18.702105 8678 heartbeater.cc:347] Connected to a master server at 127.10.184.126:41491
I20260809 08:51:18.702203 8678 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:18.702361 8678 heartbeater.cc:511] Master 127.10.184.126:41491 requested a full tablet report, sending...
I20260809 08:51:18.702759 8022 ts_manager.cc:194] Registered new tserver with Master: c451fe718da04637879e376572867c7f (127.10.184.67:45673)
I20260809 08:51:18.703105 8022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.10.184.67@KRBTEST.COM'} at 127.10.184.67:33301
I20260809 08:51:18.708098 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:18.714509 8084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.709381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.10.184.126:41491)
0809 08:51:18.709427 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.709429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.709524 (+ 95us) server_negotiation.cc:407] Connection header received
0809 08:51:18.709586 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.709587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.709596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.709640 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.710147 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.710605 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.711299 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.711482 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.711534 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.712526 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.712527 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.712528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:18.712535 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.713613 (+ 1078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.713932 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.713933 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.713933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.713978 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:18.714103 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:18.714103 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:18.714103 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:18.714235 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.714306 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.714379 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.714441 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 08:51:18.715795 8022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47554:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:18.717489 8022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:18.723440 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:35745 (local address 127.0.0.1:52148)
0809 08:51:18.719677 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.719713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.719776 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.720068 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.720069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.720071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:18.720123 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.720125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721423 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.721424 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.722364 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.722366 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723154 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.723234 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723249 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.723316 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.723366 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":29,"spinlock_wait_cycles":384,"thread_start_us":49,"threads_started":1}
I20260809 08:51:18.723564 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43639 (local address 127.0.0.1:53626)
0809 08:51:18.719677 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.719713 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.719776 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.719959 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.719962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.719965 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:18.720027 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.720029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721413 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.721413 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.722621 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.722624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723338 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.723398 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723403 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.723450 (+ 47us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.723490 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":9,"thread_start_us":50,"threads_started":1}
I20260809 08:51:18.723672 8694 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:45673 (local address 127.0.0.1:59224)
0809 08:51:18.720036 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.720046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.720093 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.720369 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.720370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.720372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:18.720409 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.720410 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721307 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.721308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.722636 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.722638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723445 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723447 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.723501 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723506 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.723568 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.723603 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":29,"thread_start_us":38,"threads_started":1}
I20260809 08:51:18.723697 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.10.184.67:45673)
0809 08:51:18.719993 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.719997 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.720171 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:51:18.720212 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.720214 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.720229 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.720291 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:18.720464 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721217 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.722712 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723375 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723442 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723494 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.723584 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.723625 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":65,"threads_started":1}
I20260809 08:51:18.723755 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.10.184.65:35745)
0809 08:51:18.719709 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.719714 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.719858 (+ 144us) server_negotiation.cc:407] Connection header received
0809 08:51:18.719905 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.719907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.719923 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.719988 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:18.720214 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721326 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.722433 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723060 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723132 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723185 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.723632 (+ 447us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.723682 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260809 08:51:18.723886 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.719480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.10.184.66:43639)
0809 08:51:18.719797 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.719801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.719811 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:18.719849 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.719850 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.719863 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.719940 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:18.720111 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.721306 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.722729 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.723250 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.723325 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.723382 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.723755 (+ 373us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.723801 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":87,"threads_started":1}
I20260809 08:51:18.724512 8591 tablet_service.cc:1467] Processing CreateTablet for tablet 9be1a755f3a1495eb3921720478c8f91 (DEFAULT_TABLE table=test-table [id=e811c32bafe548a8b84e2fcfe1303931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:18.724740 8400 tablet_service.cc:1467] Processing CreateTablet for tablet 9be1a755f3a1495eb3921720478c8f91 (DEFAULT_TABLE table=test-table [id=e811c32bafe548a8b84e2fcfe1303931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:18.724860 8591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be1a755f3a1495eb3921720478c8f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:18.725052 8400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be1a755f3a1495eb3921720478c8f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:18.725319 8208 tablet_service.cc:1467] Processing CreateTablet for tablet 9be1a755f3a1495eb3921720478c8f91 (DEFAULT_TABLE table=test-table [id=e811c32bafe548a8b84e2fcfe1303931]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:18.725617 8208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9be1a755f3a1495eb3921720478c8f91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:18.727684 8696 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Bootstrap starting.
I20260809 08:51:18.728157 8698 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Bootstrap starting.
I20260809 08:51:18.728358 8696 tablet_bootstrap.cc:654] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:18.728991 8698 tablet_bootstrap.cc:654] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:18.728978 8696 log.cc:824] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:18.729061 8700 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: Bootstrap starting.
I20260809 08:51:18.729548 8698 log.cc:824] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:18.729692 8700 tablet_bootstrap.cc:654] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:18.729765 8696 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: No bootstrap required, opened a new log
I20260809 08:51:18.729833 8696 ts_tablet_manager.cc:1397] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:51:18.730170 8700 log.cc:824] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:18.730777 8700 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: No bootstrap required, opened a new log
I20260809 08:51:18.730835 8700 ts_tablet_manager.cc:1397] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:18.731200 8696 raft_consensus.cc:348] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.731333 8696 raft_consensus.cc:374] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:18.731354 8696 raft_consensus.cc:729] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33a4df817ff940bcbe9473753e9184b4, State: Initialized, Role: FOLLOWER
I20260809 08:51:18.731482 8696 consensus_queue.cc:260] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.731721 8696 ts_tablet_manager.cc:1428] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 08:51:18.731772 8496 heartbeater.cc:503] Master 127.10.184.126:41491 was elected leader, sending a full tablet report...
I20260809 08:51:18.732272 8698 tablet_bootstrap.cc:492] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: No bootstrap required, opened a new log
I20260809 08:51:18.732326 8698 ts_tablet_manager.cc:1397] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 08:51:18.733014 8700 raft_consensus.cc:348] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.733145 8700 raft_consensus.cc:374] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:18.733187 8700 raft_consensus.cc:729] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1363ec5b0cca41faaefe5dd256da0f4c, State: Initialized, Role: FOLLOWER
I20260809 08:51:18.733292 8700 consensus_queue.cc:260] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.733554 8305 heartbeater.cc:503] Master 127.10.184.126:41491 was elected leader, sending a full tablet report...
I20260809 08:51:18.733765 8700 ts_tablet_manager.cc:1428] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 08:51:18.733940 8698 raft_consensus.cc:348] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.734103 8698 raft_consensus.cc:374] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:18.734148 8698 raft_consensus.cc:729] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c451fe718da04637879e376572867c7f, State: Initialized, Role: FOLLOWER
I20260809 08:51:18.734256 8698 consensus_queue.cc:260] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.734508 8698 ts_tablet_manager.cc:1428] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:18.734519 8678 heartbeater.cc:503] Master 127.10.184.126:41491 was elected leader, sending a full tablet report...
I20260809 08:51:18.749537 8707 raft_consensus.cc:482] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:18.749612 8707 raft_consensus.cc:504] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.749867 8707 leader_election.cc:288] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 33a4df817ff940bcbe9473753e9184b4 (127.10.184.66:43639), c451fe718da04637879e376572867c7f (127.10.184.67:45673)
I20260809 08:51:18.753216 8695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.750153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:59293 (local address 127.10.184.67:45673)
0809 08:51:18.750189 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.750191 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.750280 (+ 89us) server_negotiation.cc:407] Connection header received
0809 08:51:18.750310 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.750311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.750320 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.750367 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:18.750506 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.751170 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752354 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.752925 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752982 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.752989 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.753115 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.753152 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:18.753216 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.750060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:40429 (local address 127.10.184.66:43639)
0809 08:51:18.750109 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.750111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.750118 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:18.750142 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.750142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.750165 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.750221 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:18.750351 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.751040 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752316 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.752879 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752974 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.752983 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.753115 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.753161 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 08:51:18.753245 8308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.749991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:43639 (local address 127.10.184.65:40429)
0809 08:51:18.750035 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.750047 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.750085 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.750230 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.750231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.750233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:18.750292 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.750294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.751105 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.751106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.752242 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.752245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.752967 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.753044 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.753052 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.753105 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.753151 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":6}
I20260809 08:51:18.753252 8709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.750082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:45673 (local address 127.10.184.65:59293)
0809 08:51:18.750218 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.750226 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:18.750274 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:18.750373 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:18.750373 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:18.750374 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:18.750439 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.750441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.751232 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.751233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.752291 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:18.752293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.752982 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.752982 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:18.753044 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.753051 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 08:51:18.753105 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 08:51:18.753151 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260809 08:51:18.753525 8425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be1a755f3a1495eb3921720478c8f91" candidate_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a4df817ff940bcbe9473753e9184b4" is_pre_election: true
I20260809 08:51:18.753659 8425 raft_consensus.cc:2454] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1363ec5b0cca41faaefe5dd256da0f4c in term 0.
I20260809 08:51:18.753711 8614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be1a755f3a1495eb3921720478c8f91" candidate_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c451fe718da04637879e376572867c7f" is_pre_election: true
I20260809 08:51:18.753827 8614 raft_consensus.cc:2454] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1363ec5b0cca41faaefe5dd256da0f4c in term 0.
I20260809 08:51:18.753885 8139 leader_election.cc:302] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1363ec5b0cca41faaefe5dd256da0f4c, 33a4df817ff940bcbe9473753e9184b4; no voters:
I20260809 08:51:18.754046 8707 raft_consensus.cc:2781] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:18.754087 8707 raft_consensus.cc:482] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:18.754102 8707 raft_consensus.cc:3037] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:18.754715 8707 raft_consensus.cc:504] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.754822 8707 leader_election.cc:288] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [CANDIDATE]: Term 1 election: Requested vote from peers 33a4df817ff940bcbe9473753e9184b4 (127.10.184.66:43639), c451fe718da04637879e376572867c7f (127.10.184.67:45673)
I20260809 08:51:18.754957 8614 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be1a755f3a1495eb3921720478c8f91" candidate_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c451fe718da04637879e376572867c7f"
I20260809 08:51:18.754968 8425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9be1a755f3a1495eb3921720478c8f91" candidate_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33a4df817ff940bcbe9473753e9184b4"
I20260809 08:51:18.755025 8425 raft_consensus.cc:3037] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:18.755024 8614 raft_consensus.cc:3037] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:18.755774 8614 raft_consensus.cc:2454] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1363ec5b0cca41faaefe5dd256da0f4c in term 1.
I20260809 08:51:18.755811 8425 raft_consensus.cc:2454] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1363ec5b0cca41faaefe5dd256da0f4c in term 1.
I20260809 08:51:18.755934 8139 leader_election.cc:302] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1363ec5b0cca41faaefe5dd256da0f4c, 33a4df817ff940bcbe9473753e9184b4; no voters:
I20260809 08:51:18.756028 8707 raft_consensus.cc:2781] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:18.756151 8707 raft_consensus.cc:686] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [term 1 LEADER]: Becoming Leader. State: Replica: 1363ec5b0cca41faaefe5dd256da0f4c, State: Running, Role: LEADER
I20260809 08:51:18.756223 8707 consensus_queue.cc:237] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } }
I20260809 08:51:18.756954 8022 catalog_manager.cc:5495] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1363ec5b0cca41faaefe5dd256da0f4c (127.10.184.65). New cstate: current_term: 1 leader_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1363ec5b0cca41faaefe5dd256da0f4c" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 35745 } health_report { overall_health: HEALTHY } } }
I20260809 08:51:18.768450 8693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.766556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52162 (local address 127.10.184.65:35745)
0809 08:51:18.766597 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.766599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.766606 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:18.766664 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.766665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.766680 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.766732 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:18.766821 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.767245 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.767901 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.768072 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.768116 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.768217 (+ 101us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:18.768298 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:18.768347 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.768360 (+ 13us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.768375 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 08:51:18.769963 8425 raft_consensus.cc:1261] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 1 FOLLOWER]: Refusing update from remote peer 1363ec5b0cca41faaefe5dd256da0f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:18.769963 8614 raft_consensus.cc:1261] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 1 FOLLOWER]: Refusing update from remote peer 1363ec5b0cca41faaefe5dd256da0f4c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:18.770284 8710 consensus_queue.cc:1035] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "33a4df817ff940bcbe9473753e9184b4" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 43639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:18.770393 8707 consensus_queue.cc:1035] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c451fe718da04637879e376572867c7f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 45673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:18.773327 8718 mvcc.cc:205] Tried to move back new op lower bound from 7316543401039589376 to 7316543400986013696. Current Snapshot: MvccSnapshot[applied={T|T < 7316543401039589376}]
I20260809 08:51:18.774387 8719 mvcc.cc:205] Tried to move back new op lower bound from 7316543401039589376 to 7316543400986013696. Current Snapshot: MvccSnapshot[applied={T|T < 7316543401039589376 or (T in {7316543401039589376})}]
I20260809 08:51:18.774672 8021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:18.774748 8021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47554:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:18.775388 8021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da321fe0ca104aaeaac8f8450363e68c: Sending DeleteTablet for 3 replicas of tablet 9be1a755f3a1495eb3921720478c8f91
I20260809 08:51:18.775689 8208 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be1a755f3a1495eb3921720478c8f91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:18 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:52148
I20260809 08:51:18.775727 8400 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be1a755f3a1495eb3921720478c8f91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:18 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:53626
I20260809 08:51:18.775718 8591 tablet_service.cc:1514] Processing DeleteTablet for tablet 9be1a755f3a1495eb3921720478c8f91 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:18 UTC) from {username='kudu', principal='kudu/127.10.184.126@KRBTEST.COM'} at 127.0.0.1:59224
I20260809 08:51:18.775916 8733 tablet_replica.cc:330] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: stopping tablet replica
I20260809 08:51:18.775986 8731 tablet_replica.cc:330] T 9be1a755f3a1495eb3921720478c8f91 P 1363ec5b0cca41faaefe5dd256da0f4c: stopping tablet replica
I20260809 08:51:18.776031 8733 raft_consensus.cc:2229] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:18.776106 8730 tablet_replica.cc:330] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: stopping tablet replica
I20260809 08:51:18.776170 8733 raft_consensus.cc:2258] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:18.776211 8730 raft_consensus.cc:2229] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:18.776239 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 8112
I20260809 08:51:18.776314 8730 raft_consensus.cc:2258] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:18.776927 8733 ts_tablet_manager.cc:1905] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:18.777042 8730 ts_tablet_manager.cc:1905] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:18.779232 8730 ts_tablet_manager.cc:1918] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:18.779239 8733 ts_tablet_manager.cc:1918] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:18.779297 8733 log.cc:1196] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/9be1a755f3a1495eb3921720478c8f91
I20260809 08:51:18.779297 8730 log.cc:1196] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/9be1a755f3a1495eb3921720478c8f91
I20260809 08:51:18.779559 8730 ts_tablet_manager.cc:1939] T 9be1a755f3a1495eb3921720478c8f91 P 33a4df817ff940bcbe9473753e9184b4: Deleting consensus metadata
I20260809 08:51:18.779558 8733 ts_tablet_manager.cc:1939] T 9be1a755f3a1495eb3921720478c8f91 P c451fe718da04637879e376572867c7f: Deleting consensus metadata
I20260809 08:51:18.780697 8001 catalog_manager.cc:4841] TS 33a4df817ff940bcbe9473753e9184b4 (127.10.184.66:43639): tablet 9be1a755f3a1495eb3921720478c8f91 (table test-table [id=e811c32bafe548a8b84e2fcfe1303931]) successfully deleted
I20260809 08:51:18.780783 8001 catalog_manager.cc:4841] TS c451fe718da04637879e376572867c7f (127.10.184.67:45673): tablet 9be1a755f3a1495eb3921720478c8f91 (table test-table [id=e811c32bafe548a8b84e2fcfe1303931]) successfully deleted
W20260809 08:51:18.784907 7999 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:35745 (error 108)
I20260809 08:51:18.785130 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 8313
I20260809 08:51:18.787277 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.785233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:35745 (local address 127.0.0.1:52170)
0809 08:51:18.787211 (+ 1978us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:18.787249 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:35745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1959}
W20260809 08:51:18.792515 7999 catalog_manager.cc:4584] TS 1363ec5b0cca41faaefe5dd256da0f4c (127.10.184.65:35745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9be1a755f3a1495eb3921720478c8f91: Network error: Client connection negotiation failed: client connection to 127.10.184.65:35745: connect: Connection refused (error 111)
I20260809 08:51:18.802769 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 8504
I20260809 08:51:18.814385 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 7973
2026-08-09T08:51:18Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (1958 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T08:51:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:18Z Disabled control of system clock
I20260809 08:51:18.845745 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39573
--webserver_interface=127.10.184.126
--webserver_port=0
--builtin_ntp_servers=127.10.184.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:18.924772 8740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:18.925000 8740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:18.925020 8740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:18.926164 8740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:18.926205 8740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:18.926216 8740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:18.926229 8740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:18.926251 8740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:18.927592 8740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:18.927820 8740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:18.928094 8740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:18.932710 8746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.932664 8749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:18.932679 8747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:18.932832 8740 server_base.cc:1034] running on GCE node
I20260809 08:51:18.933233 8740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:18.933497 8740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:18.934628 8740 hybrid_clock.cc:648] HybridClock initialized: now 1786265478934615 us; error 28 us; skew 500 ppm
I20260809 08:51:18.935740 8740 webserver.cc:460] Webserver started at http://127.10.184.126:46789/ using document root <none> and password file <none>
I20260809 08:51:18.935957 8740 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:18.936005 8740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:18.936127 8740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:18.937156 8740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "122577093378424492e88e3a23ce9257"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "373be6d8275dee4a61ea3702b4639eca"
server_key_iv: "b2b4188a4605c7c50ab08c858c5a5a47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.937472 8740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "122577093378424492e88e3a23ce9257"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "373be6d8275dee4a61ea3702b4639eca"
server_key_iv: "b2b4188a4605c7c50ab08c858c5a5a47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.938617 8740 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:18.939239 8756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:18.939388 8740 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:18.939482 8740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "122577093378424492e88e3a23ce9257"
format_stamp: "Formatted at 2026-08-09 08:51:18 on dist-test-slave-ncp9"
server_key: "373be6d8275dee4a61ea3702b4639eca"
server_key_iv: "b2b4188a4605c7c50ab08c858c5a5a47"
server_key_version: "encryptionkey@0"
I20260809 08:51:18.939536 8740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:18.956054 8740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:18.956272 8740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:18.956368 8740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:18.959992 8740 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39573
I20260809 08:51:18.960027 8828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39573 every 8 connection(s)
I20260809 08:51:18.960316 8740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:18.961295 8830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:18.963479 8830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: Bootstrap starting.
I20260809 08:51:18.964035 8830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:18.964313 8830 log.cc:824] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:18.964936 8830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: No bootstrap required, opened a new log
I20260809 08:51:18.966440 8830 raft_consensus.cc:348] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } }
I20260809 08:51:18.966571 8830 raft_consensus.cc:374] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:18.966602 8830 raft_consensus.cc:729] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 122577093378424492e88e3a23ce9257, State: Initialized, Role: FOLLOWER
I20260809 08:51:18.966737 8830 consensus_queue.cc:260] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } }
I20260809 08:51:18.966805 8830 raft_consensus.cc:388] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:18.966835 8830 raft_consensus.cc:482] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:18.966881 8830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:18.967382 8830 raft_consensus.cc:504] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } }
I20260809 08:51:18.967474 8830 leader_election.cc:302] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 122577093378424492e88e3a23ce9257; no voters:
I20260809 08:51:18.967629 8830 leader_election.cc:288] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:18.967669 8833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:18.967803 8833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [term 1 LEADER]: Becoming Leader. State: Replica: 122577093378424492e88e3a23ce9257, State: Running, Role: LEADER
I20260809 08:51:18.967867 8830 sys_catalog.cc:564] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:18.967902 8833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } }
I20260809 08:51:18.968261 8838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 122577093378424492e88e3a23ce9257. Latest consensus state: current_term: 1 leader_uuid: "122577093378424492e88e3a23ce9257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } } }
I20260809 08:51:18.968333 8836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "122577093378424492e88e3a23ce9257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "122577093378424492e88e3a23ce9257" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39573 } } }
I20260809 08:51:18.968421 8838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:18.968485 8836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:18.968935 8850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:18.969391 8850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:18.969888 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 8740
I20260809 08:51:18.970005 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:18.970191 10977 external_mini_cluster.cc:1428] Setting key 1d11ccf20d77c4604bc01d289e49b4e0
I20260809 08:51:18.970999 8850 catalog_manager.cc:1347] Generated new cluster ID: b506d96ba4f54f79a458ad6269e92ba8
I20260809 08:51:18.971041 8850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:18.976140 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:18.970557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36868 (local address 127.10.184.126:39573)
0809 08:51:18.970737 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:18.970744 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:18.971190 (+ 446us) server_negotiation.cc:407] Connection header received
0809 08:51:18.971308 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:18.971315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:18.971379 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:18.971504 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:18.971727 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.973070 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.974142 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:18.974734 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:18.974840 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:18.975498 (+ 658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:18.975502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:18.975503 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:18.975697 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:18.975735 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:18.975749 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:18.975802 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:18.975885 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 08:51:18.975957 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":89,"threads_started":1}
I20260809 08:51:18.991776 8850 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:18.992269 8850 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:18.995249 8850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: Generated new TSK 0
I20260809 08:51:18.995400 8850 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:19.002944 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.65:0
--local_ip_for_outbound_sockets=127.10.184.65
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--builtin_ntp_servers=127.10.184.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:19.084067 8868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:19.084245 8868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:19.084265 8868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:19.085381 8868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:19.085417 8868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:19.085441 8868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.65
I20260809 08:51:19.086736 8868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.10.184.65
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.65
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:19.086936 8868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:19.087195 8868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:19.091970 8877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:19.092056 8873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:19.092021 8875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:19.092201 8868 server_base.cc:1034] running on GCE node
I20260809 08:51:19.092394 8868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:19.092610 8868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:19.093755 8868 hybrid_clock.cc:648] HybridClock initialized: now 1786265479093740 us; error 25 us; skew 500 ppm
I20260809 08:51:19.094885 8868 webserver.cc:460] Webserver started at http://127.10.184.65:41981/ using document root <none> and password file <none>
I20260809 08:51:19.095096 8868 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:19.095144 8868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:19.095275 8868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:19.096247 8868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data/instance:
uuid: "93b2cb63715b45f98b39e480ba3c6d59"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "8596319fa96dc7915efec4352278701d"
server_key_iv: "19f1ca85be770cb7a55536976e49541c"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.096580 8868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance:
uuid: "93b2cb63715b45f98b39e480ba3c6d59"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "8596319fa96dc7915efec4352278701d"
server_key_iv: "19f1ca85be770cb7a55536976e49541c"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.097836 8868 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:19.098477 8886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.098619 8868 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:19.098708 8868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal
uuid: "93b2cb63715b45f98b39e480ba3c6d59"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "8596319fa96dc7915efec4352278701d"
server_key_iv: "19f1ca85be770cb7a55536976e49541c"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.098774 8868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:19.114440 8868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:19.114688 8868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:19.114809 8868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:19.114996 8868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:19.115274 8868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:19.115306 8868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.115332 8868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:19.115350 8868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.120697 8868 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.65:44855
I20260809 08:51:19.120759 9040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.65:44855 every 8 connection(s)
I20260809 08:51:19.121057 8868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/data/info.pb
I20260809 08:51:19.125607 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.121740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.65:38483 (local address 127.10.184.126:39573)
0809 08:51:19.121789 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.121792 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.121962 (+ 170us) server_negotiation.cc:407] Connection header received
0809 08:51:19.122426 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.122427 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.122439 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.122474 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:19.123068 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.123550 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.124429 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.124605 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.124651 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.125400 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:19.125401 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:19.125402 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:19.125409 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:19.125422 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:19.125426 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:19.125469 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.125525 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.125548 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:19.125764 9046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.121701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39573 (local address 127.10.184.65:38483)
0809 08:51:19.121861 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.121891 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.122342 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.122530 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.122533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.122541 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:19.123011 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.123015 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.123609 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.123609 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.124373 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.124374 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.124654 (+ 280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.124654 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.124750 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.125278 (+ 528us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:19.125278 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:19.125306 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:19.125327 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:19.125473 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:19.125475 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:19.125477 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.125520 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.125594 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260809 08:51:19.126237 9044 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39573
I20260809 08:51:19.126317 9044 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:19.126452 9044 heartbeater.cc:511] Master 127.10.184.126:39573 requested a full tablet report, sending...
I20260809 08:51:19.126892 8779 ts_manager.cc:194] Registered new tserver with Master: 93b2cb63715b45f98b39e480ba3c6d59 (127.10.184.65:44855)
I20260809 08:51:19.127478 8779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.65:38483
I20260809 08:51:19.128186 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 8868
I20260809 08:51:19.128300 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-0/wal/instance
I20260809 08:51:19.128468 10977 external_mini_cluster.cc:1428] Setting key afbc1bb58347edbb74d4ee1f08525a37
I20260809 08:51:19.129283 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.66:0
--local_ip_for_outbound_sockets=127.10.184.66
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--builtin_ntp_servers=127.10.184.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:19.207574 9047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:19.207767 9047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:19.207788 9047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:19.208926 9047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:19.208969 9047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:19.208997 9047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.66
I20260809 08:51:19.210310 9047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.10.184.66
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.66
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:19.210525 9047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:19.210805 9047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:19.215512 9047 server_base.cc:1034] running on GCE node
W20260809 08:51:19.215464 9056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:19.215483 9058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:19.215449 9055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:19.215893 9047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:19.216147 9047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:19.217288 9047 hybrid_clock.cc:648] HybridClock initialized: now 1786265479217268 us; error 30 us; skew 500 ppm
I20260809 08:51:19.218438 9047 webserver.cc:460] Webserver started at http://127.10.184.66:42275/ using document root <none> and password file <none>
I20260809 08:51:19.218655 9047 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:19.218704 9047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:19.218842 9047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:19.219820 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data/instance:
uuid: "ecf5e0fa67aa495d8b0122be777f925f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "438742923a18ac2dc1f361eeb18bcd50"
server_key_iv: "2b4ff9463c435c3b0382fc930cd62d1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.220139 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance:
uuid: "ecf5e0fa67aa495d8b0122be777f925f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "438742923a18ac2dc1f361eeb18bcd50"
server_key_iv: "2b4ff9463c435c3b0382fc930cd62d1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.221364 9047 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:19.221979 9068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.222110 9047 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:19.222229 9047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal
uuid: "ecf5e0fa67aa495d8b0122be777f925f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "438742923a18ac2dc1f361eeb18bcd50"
server_key_iv: "2b4ff9463c435c3b0382fc930cd62d1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.222288 9047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:19.230979 9047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:19.231215 9047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:19.231325 9047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:19.231498 9047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:19.231772 9047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:19.231812 9047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.231838 9047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:19.231863 9047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.237135 9047 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.66:35281
I20260809 08:51:19.237180 9220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.66:35281 every 8 connection(s)
I20260809 08:51:19.237455 9047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/data/info.pb
I20260809 08:51:19.242105 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.238087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.66:50387 (local address 127.10.184.126:39573)
0809 08:51:19.238143 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.238145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.238431 (+ 286us) server_negotiation.cc:407] Connection header received
0809 08:51:19.238894 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.238895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.238905 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.238954 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:19.239584 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.240058 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.240977 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.241163 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.241203 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.241915 (+ 712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:19.241915 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:19.241916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:19.241919 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:19.241928 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:19.241931 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:19.241959 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.242032 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.242048 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:51:19.242309 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.238048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39573 (local address 127.10.184.66:50387)
0809 08:51:19.238241 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.238278 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.238817 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.239006 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.239009 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.239017 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:19.239521 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.239526 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.240119 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.240119 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.240906 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.240909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.241216 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.241216 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.241324 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.241805 (+ 481us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:19.241805 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:19.241832 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:19.241853 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:19.241965 (+ 112us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:19.241968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:19.241972 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.242025 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.242100 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260809 08:51:19.242786 9222 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39573
I20260809 08:51:19.242871 9222 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:19.243026 9222 heartbeater.cc:511] Master 127.10.184.126:39573 requested a full tablet report, sending...
I20260809 08:51:19.243360 8779 ts_manager.cc:194] Registered new tserver with Master: ecf5e0fa67aa495d8b0122be777f925f (127.10.184.66:35281)
I20260809 08:51:19.243698 8779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.66:50387
I20260809 08:51:19.243914 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9047
I20260809 08:51:19.244024 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal/instance
I20260809 08:51:19.244201 10977 external_mini_cluster.cc:1428] Setting key 69ad68b810328607ebd94bc49ba1e77a
I20260809 08:51:19.245018 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.10.184.67:0
--local_ip_for_outbound_sockets=127.10.184.67
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--builtin_ntp_servers=127.10.184.84:40409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 08:51:19.323318 9225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:19.323503 9225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:19.323529 9225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:19.324644 9225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:19.324684 9225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:19.324717 9225 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.10.184.67
I20260809 08:51:19.326077 9225 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.10.184.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.10.184.67
--webserver_port=0
--tserver_master_addrs=127.10.184.126:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.10.184.67
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:19.326303 9225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:19.326555 9225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:19.331146 9231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:19.331156 9233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:19.331311 9225 server_base.cc:1034] running on GCE node
W20260809 08:51:19.331168 9235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:19.331605 9225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:19.331840 9225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:19.332940 9225 hybrid_clock.cc:648] HybridClock initialized: now 1786265479332927 us; error 32 us; skew 500 ppm
I20260809 08:51:19.334017 9225 webserver.cc:460] Webserver started at http://127.10.184.67:39129/ using document root <none> and password file <none>
I20260809 08:51:19.334227 9225 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:19.334275 9225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:19.334396 9225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:19.335359 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data/instance:
uuid: "c846dbc7bba54862a206c4d2f639966f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "a087866b7fb56ca97fc2253a6333acdc"
server_key_iv: "11250df84f9cfbadf82da0b9618a1e63"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.335686 9225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance:
uuid: "c846dbc7bba54862a206c4d2f639966f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "a087866b7fb56ca97fc2253a6333acdc"
server_key_iv: "11250df84f9cfbadf82da0b9618a1e63"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.336935 9225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:19.337580 9241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.337724 9225 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:19.337806 9225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal
uuid: "c846dbc7bba54862a206c4d2f639966f"
format_stamp: "Formatted at 2026-08-09 08:51:19 on dist-test-slave-ncp9"
server_key: "a087866b7fb56ca97fc2253a6333acdc"
server_key_iv: "11250df84f9cfbadf82da0b9618a1e63"
server_key_version: "encryptionkey@0"
I20260809 08:51:19.337872 9225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:19.350407 9225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:19.350633 9225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:19.350754 9225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:19.350936 9225 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 08:51:19.351220 9225 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 08:51:19.351259 9225 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.351285 9225 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 08:51:19.351302 9225 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:19.356348 9225 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.67:33515
I20260809 08:51:19.356412 9395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.67:33515 every 8 connection(s)
I20260809 08:51:19.356659 9225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/data/info.pb
I20260809 08:51:19.359632 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9225
I20260809 08:51:19.359726 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal/instance
I20260809 08:51:19.359879 10977 external_mini_cluster.cc:1428] Setting key 8aadac41559f468355e80f10491986f6
I20260809 08:51:19.361454 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.357331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:35679 (local address 127.10.184.126:39573)
0809 08:51:19.357393 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.357395 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.357580 (+ 185us) server_negotiation.cc:407] Connection header received
0809 08:51:19.358002 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.358003 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.358012 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.358059 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:19.358635 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.359115 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.360159 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.360442 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.360508 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.361259 (+ 751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:19.361261 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:19.361261 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:19.361265 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:19.361279 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:19.361283 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:19.361323 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.361375 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.361398 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:51:19.361637 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.357305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39573 (local address 127.10.184.67:35679)
0809 08:51:19.357471 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.357507 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.357930 (+ 423us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.358108 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.358111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.358119 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:19.358574 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.358578 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.359174 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.359175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.360057 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.360061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.360525 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.360526 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.360681 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.361140 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 08:51:19.361140 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:19.361168 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 08:51:19.361189 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:19.361329 (+ 140us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:19.361331 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:19.361334 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.361375 (+ 41us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.361448 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260809 08:51:19.362053 9396 heartbeater.cc:347] Connected to a master server at 127.10.184.126:39573
I20260809 08:51:19.362128 9396 heartbeater.cc:464] Registering TS with master...
I20260809 08:51:19.362303 9396 heartbeater.cc:511] Master 127.10.184.126:39573 requested a full tablet report, sending...
I20260809 08:51:19.362625 8779 ts_manager.cc:194] Registered new tserver with Master: c846dbc7bba54862a206c4d2f639966f (127.10.184.67:33515)
I20260809 08:51:19.362943 8779 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.10.184.67:35679
I20260809 08:51:19.371315 10977 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 08:51:19.375176 8867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.372545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36874 (local address 127.10.184.126:39573)
0809 08:51:19.372603 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.372605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.372715 (+ 110us) server_negotiation.cc:407] Connection header received
0809 08:51:19.372761 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.372762 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.372774 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.372818 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:19.372980 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.373436 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.374160 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.374343 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.374396 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.374959 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:19.374961 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:19.374961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:19.374965 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:19.374978 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:19.374982 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:19.375026 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.375078 (+ 52us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.375098 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 08:51:19.376564 8779 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36874:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 08:51:19.378013 8779 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 08:51:19.383994 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.380178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38900 (local address 127.10.184.67:33515)
0809 08:51:19.380604 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.380607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.380616 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:19.380644 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.380645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.380659 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.380730 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:19.380823 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.381889 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.382915 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.383620 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.383707 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.383786 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.383892 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.383923 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":79,"threads_started":1}
I20260809 08:51:19.384001 9404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.379880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.67:33515 (local address 127.0.0.1:38900)
0809 08:51:19.380148 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.380158 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.380228 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.380729 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.380731 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.380733 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:19.380769 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.380771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.381960 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.381961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.382846 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.382849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.383715 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.383715 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.383794 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.383807 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.383876 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.383925 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":89,"threads_started":1}
I20260809 08:51:19.384793 9405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.379863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:35281 (local address 127.0.0.1:49032)
0809 08:51:19.380023 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.380063 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.380126 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.380526 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.380526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.380528 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:19.380586 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.380588 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.382820 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.382821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.383547 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.383550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.384591 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.384592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.384641 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.384643 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.384691 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.384728 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":1,"thread_start_us":61,"threads_started":1}
I20260809 08:51:19.384825 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.379972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49032 (local address 127.10.184.66:35281)
0809 08:51:19.380384 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.380387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.380395 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:19.380429 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.380431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.380446 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.380519 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:19.381997 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.382758 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.383625 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.384333 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.384391 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.384473 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.384718 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.384753 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":82,"threads_started":1}
I20260809 08:51:19.385030 9305 tablet_service.cc:1467] Processing CreateTablet for tablet 920184f82afe4716b0e34659871b9275 (DEFAULT_TABLE table=test-table [id=b83cc9496ca3403a863de9000c684927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:19.385224 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.380063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:44855 (local address 127.0.0.1:45890)
0809 08:51:19.380303 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.380311 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.380343 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.380513 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.380517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.380519 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:19.380586 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.380589 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.382924 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.382925 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.383957 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.383961 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.384984 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.384986 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.385051 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.385055 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.385111 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.385150 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":3,"thread_start_us":64,"threads_started":1}
I20260809 08:51:19.385267 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.380165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45890 (local address 127.10.184.65:44855)
0809 08:51:19.380333 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.380337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.380348 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:51:19.380405 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.380407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.380421 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.380498 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:19.380713 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.381890 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.384055 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.384729 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.384795 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.384947 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.385156 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.385191 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":95,"threads_started":1}
I20260809 08:51:19.385408 9305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920184f82afe4716b0e34659871b9275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:19.387718 9412 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Bootstrap starting.
I20260809 08:51:19.388379 9412 tablet_bootstrap.cc:654] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:19.388356 9133 tablet_service.cc:1467] Processing CreateTablet for tablet 920184f82afe4716b0e34659871b9275 (DEFAULT_TABLE table=test-table [id=b83cc9496ca3403a863de9000c684927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:19.388437 8951 tablet_service.cc:1467] Processing CreateTablet for tablet 920184f82afe4716b0e34659871b9275 (DEFAULT_TABLE table=test-table [id=b83cc9496ca3403a863de9000c684927]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 08:51:19.388640 9133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920184f82afe4716b0e34659871b9275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:19.388742 9412 log.cc:824] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:19.388778 8951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 920184f82afe4716b0e34659871b9275. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:19.389446 9412 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: No bootstrap required, opened a new log
I20260809 08:51:19.389521 9412 ts_tablet_manager.cc:1397] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260809 08:51:19.391268 9419 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: Bootstrap starting.
I20260809 08:51:19.391969 9419 tablet_bootstrap.cc:654] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:19.392053 9412 raft_consensus.cc:348] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.392203 9420 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Bootstrap starting.
I20260809 08:51:19.392228 9412 raft_consensus.cc:374] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:19.392263 9412 raft_consensus.cc:729] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c846dbc7bba54862a206c4d2f639966f, State: Initialized, Role: FOLLOWER
I20260809 08:51:19.392318 9419 log.cc:824] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:19.392422 9412 consensus_queue.cc:260] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.392807 9420 tablet_bootstrap.cc:654] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:19.392807 9412 ts_tablet_manager.cc:1428] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 08:51:19.392830 9396 heartbeater.cc:503] Master 127.10.184.126:39573 was elected leader, sending a full tablet report...
I20260809 08:51:19.393110 9420 log.cc:824] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:19.393580 9419 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: No bootstrap required, opened a new log
I20260809 08:51:19.393657 9419 ts_tablet_manager.cc:1397] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:19.393857 9420 tablet_bootstrap.cc:492] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: No bootstrap required, opened a new log
I20260809 08:51:19.393921 9420 ts_tablet_manager.cc:1397] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 08:51:19.395630 9419 raft_consensus.cc:348] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.395757 9419 raft_consensus.cc:374] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:19.395776 9419 raft_consensus.cc:729] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b2cb63715b45f98b39e480ba3c6d59, State: Initialized, Role: FOLLOWER
I20260809 08:51:19.395854 9419 consensus_queue.cc:260] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.395875 9420 raft_consensus.cc:348] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.396014 9420 raft_consensus.cc:374] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:19.396054 9420 raft_consensus.cc:729] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecf5e0fa67aa495d8b0122be777f925f, State: Initialized, Role: FOLLOWER
I20260809 08:51:19.396075 9419 ts_tablet_manager.cc:1428] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:19.396136 9044 heartbeater.cc:503] Master 127.10.184.126:39573 was elected leader, sending a full tablet report...
I20260809 08:51:19.396174 9420 consensus_queue.cc:260] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.396417 9420 ts_tablet_manager.cc:1428] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 08:51:19.396469 9222 heartbeater.cc:503] Master 127.10.184.126:39573 was elected leader, sending a full tablet report...
I20260809 08:51:19.443447 9422 raft_consensus.cc:482] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 08:51:19.443590 9422 raft_consensus.cc:504] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.443985 9422 leader_election.cc:288] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ecf5e0fa67aa495d8b0122be777f925f (127.10.184.66:35281), 93b2cb63715b45f98b39e480ba3c6d59 (127.10.184.65:44855)
I20260809 08:51:19.447889 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.444141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.66:35281 (local address 127.10.184.67:32893)
0809 08:51:19.444377 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.444386 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.444436 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.444587 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.444588 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.444590 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:19.444669 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.444671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.445564 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.445566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.446711 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.446714 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.447632 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.447633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.447710 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.447713 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.447768 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.447814 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219}
I20260809 08:51:19.447903 9406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.444204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:32893 (local address 127.10.184.66:35281)
0809 08:51:19.444266 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.444269 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.444446 (+ 177us) server_negotiation.cc:407] Connection header received
0809 08:51:19.444481 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.444482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.444492 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.444576 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:19.444738 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.445483 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.446810 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.447511 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.447588 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.447598 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.447788 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.447834 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 08:51:19.448858 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.444564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.10.184.67:48295 (local address 127.10.184.65:44855)
0809 08:51:19.446169 (+ 1605us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.446170 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.446179 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:19.446205 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.446206 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.446218 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.446276 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 08:51:19.446560 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.447089 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.448183 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.448543 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.448603 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.448610 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.448759 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.448798 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1568}
I20260809 08:51:19.448868 9428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.444234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:44855 (local address 127.10.184.67:48295)
0809 08:51:19.444398 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.444409 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:19.444466 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:19.446357 (+ 1891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:19.446358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:19.446360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 08:51:19.446444 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.446446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.447153 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.447154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.448102 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:19.448105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.448623 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.448624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:19.448703 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.448707 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 08:51:19.448752 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 08:51:19.448788 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":93,"threads_started":1}
I20260809 08:51:19.449306 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920184f82afe4716b0e34659871b9275" candidate_uuid: "c846dbc7bba54862a206c4d2f639966f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93b2cb63715b45f98b39e480ba3c6d59" is_pre_election: true
I20260809 08:51:19.449435 8980 raft_consensus.cc:2454] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c846dbc7bba54862a206c4d2f639966f in term 0.
I20260809 08:51:19.449615 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920184f82afe4716b0e34659871b9275" candidate_uuid: "c846dbc7bba54862a206c4d2f639966f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecf5e0fa67aa495d8b0122be777f925f" is_pre_election: true
I20260809 08:51:19.449653 9244 leader_election.cc:302] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93b2cb63715b45f98b39e480ba3c6d59, c846dbc7bba54862a206c4d2f639966f; no voters:
I20260809 08:51:19.449744 9158 raft_consensus.cc:2454] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c846dbc7bba54862a206c4d2f639966f in term 0.
I20260809 08:51:19.449777 9422 raft_consensus.cc:2781] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 08:51:19.449832 9422 raft_consensus.cc:482] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 08:51:19.449855 9422 raft_consensus.cc:3037] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:19.450541 9422 raft_consensus.cc:504] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.450655 9422 leader_election.cc:288] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [CANDIDATE]: Term 1 election: Requested vote from peers ecf5e0fa67aa495d8b0122be777f925f (127.10.184.66:35281), 93b2cb63715b45f98b39e480ba3c6d59 (127.10.184.65:44855)
I20260809 08:51:19.450837 8980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920184f82afe4716b0e34659871b9275" candidate_uuid: "c846dbc7bba54862a206c4d2f639966f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "93b2cb63715b45f98b39e480ba3c6d59"
I20260809 08:51:19.450848 9158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "920184f82afe4716b0e34659871b9275" candidate_uuid: "c846dbc7bba54862a206c4d2f639966f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ecf5e0fa67aa495d8b0122be777f925f"
I20260809 08:51:19.450911 9158 raft_consensus.cc:3037] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:19.450909 8980 raft_consensus.cc:3037] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:19.451591 9158 raft_consensus.cc:2454] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c846dbc7bba54862a206c4d2f639966f in term 1.
I20260809 08:51:19.451607 8980 raft_consensus.cc:2454] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c846dbc7bba54862a206c4d2f639966f in term 1.
I20260809 08:51:19.451753 9244 leader_election.cc:302] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 93b2cb63715b45f98b39e480ba3c6d59, c846dbc7bba54862a206c4d2f639966f; no voters:
I20260809 08:51:19.451853 9422 raft_consensus.cc:2781] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:19.452010 9422 raft_consensus.cc:686] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 1 LEADER]: Becoming Leader. State: Replica: c846dbc7bba54862a206c4d2f639966f, State: Running, Role: LEADER
I20260809 08:51:19.452095 9422 consensus_queue.cc:237] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } }
I20260809 08:51:19.452936 8778 catalog_manager.cc:5495] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f reported cstate change: term changed from 0 to 1, leader changed from <none> to c846dbc7bba54862a206c4d2f639966f (127.10.184.67). New cstate: current_term: 1 leader_uuid: "c846dbc7bba54862a206c4d2f639966f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c846dbc7bba54862a206c4d2f639966f" member_type: VOTER last_known_addr { host: "127.10.184.67" port: 33515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 } health_report { overall_health: UNKNOWN } } }
I20260809 08:51:19.459594 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.456841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38904 (local address 127.10.184.67:33515)
0809 08:51:19.456882 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:19.456883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:19.456918 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:51:19.456989 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:19.456990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:19.457007 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:19.457077 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:19.457206 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.457729 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.458788 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:19.459063 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:19.459126 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:19.459288 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:19.459402 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:19.459478 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:19.459499 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 08:51:19.459515 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:19.461210 9158 raft_consensus.cc:1261] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 1 FOLLOWER]: Refusing update from remote peer c846dbc7bba54862a206c4d2f639966f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:19.461211 8980 raft_consensus.cc:1261] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 1 FOLLOWER]: Refusing update from remote peer c846dbc7bba54862a206c4d2f639966f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:19.461472 9422 consensus_queue.cc:1035] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ecf5e0fa67aa495d8b0122be777f925f" member_type: VOTER last_known_addr { host: "127.10.184.66" port: 35281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:19.461618 9429 consensus_queue.cc:1035] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [LEADER]: Connected to new peer: Peer: permanent_uuid: "93b2cb63715b45f98b39e480ba3c6d59" member_type: VOTER last_known_addr { host: "127.10.184.65" port: 44855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:19.463846 9434 mvcc.cc:205] Tried to move back new op lower bound from 7316543403871051776 to 7316543403836448768. Current Snapshot: MvccSnapshot[applied={T|T < 7316543403871051776}]
I20260809 08:51:19.464022 9435 mvcc.cc:205] Tried to move back new op lower bound from 7316543403871051776 to 7316543403836448768. Current Snapshot: MvccSnapshot[applied={T|T < 7316543403871051776}]
I20260809 08:51:19.464736 9437 mvcc.cc:205] Tried to move back new op lower bound from 7316543403871051776 to 7316543403836448768. Current Snapshot: MvccSnapshot[applied={T|T < 7316543403871051776}]
I20260809 08:51:19.465787 8778 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:19.465864 8778 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36874:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 08:51:19.466513 8778 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 122577093378424492e88e3a23ce9257: Sending DeleteTablet for 3 replicas of tablet 920184f82afe4716b0e34659871b9275
I20260809 08:51:19.466862 8951 tablet_service.cc:1514] Processing DeleteTablet for tablet 920184f82afe4716b0e34659871b9275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:19 UTC) from {username='slave'} at 127.0.0.1:45890
I20260809 08:51:19.466864 9305 tablet_service.cc:1514] Processing DeleteTablet for tablet 920184f82afe4716b0e34659871b9275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:19 UTC) from {username='slave'} at 127.0.0.1:38900
I20260809 08:51:19.466864 9133 tablet_service.cc:1514] Processing DeleteTablet for tablet 920184f82afe4716b0e34659871b9275 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 08:51:19 UTC) from {username='slave'} at 127.0.0.1:49032
I20260809 08:51:19.467092 9449 tablet_replica.cc:330] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59: stopping tablet replica
I20260809 08:51:19.467198 9449 raft_consensus.cc:2229] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:19.467289 9449 raft_consensus.cc:2258] T 920184f82afe4716b0e34659871b9275 P 93b2cb63715b45f98b39e480ba3c6d59 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:19.467319 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 8868
I20260809 08:51:19.473331 9453 tablet_replica.cc:330] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: stopping tablet replica
I20260809 08:51:19.473465 9453 raft_consensus.cc:2229] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 08:51:19.473595 9453 raft_consensus.cc:2258] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:19.479562 9453 ts_tablet_manager.cc:1905] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:19.500551 9453 ts_tablet_manager.cc:1918] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:19.500623 9455 tablet_replica.cc:330] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: stopping tablet replica
I20260809 08:51:19.500741 9453 log.cc:1196] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-1/wal/wals/920184f82afe4716b0e34659871b9275
I20260809 08:51:19.500839 9455 raft_consensus.cc:2229] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 1 LEADER]: Raft consensus shutting down.
I20260809 08:51:19.501034 9455 raft_consensus.cc:2258] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 08:51:19.501147 9453 ts_tablet_manager.cc:1939] T 920184f82afe4716b0e34659871b9275 P ecf5e0fa67aa495d8b0122be777f925f: Deleting consensus metadata
I20260809 08:51:19.501634 9455 ts_tablet_manager.cc:1905] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 08:51:19.501652 8761 catalog_manager.cc:4841] TS ecf5e0fa67aa495d8b0122be777f925f (127.10.184.66:35281): tablet 920184f82afe4716b0e34659871b9275 (table test-table [id=b83cc9496ca3403a863de9000c684927]) successfully deleted
I20260809 08:51:19.503286 9455 ts_tablet_manager.cc:1918] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 08:51:19.503361 9455 log.cc:1196] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Deleting WAL directory at /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786265241374876-10977-0/minicluster-data/ts-2/wal/wals/920184f82afe4716b0e34659871b9275
I20260809 08:51:19.503656 9455 ts_tablet_manager.cc:1939] T 920184f82afe4716b0e34659871b9275 P c846dbc7bba54862a206c4d2f639966f: Deleting consensus metadata
I20260809 08:51:19.504072 8762 catalog_manager.cc:4841] TS c846dbc7bba54862a206c4d2f639966f (127.10.184.67:33515): tablet 920184f82afe4716b0e34659871b9275 (table test-table [id=b83cc9496ca3403a863de9000c684927]) successfully deleted
W20260809 08:51:19.504973 8761 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.10.184.65:44855 (error 108)
I20260809 08:51:19.505465 9409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:19.505329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.65:44855 (local address 127.0.0.1:45892)
0809 08:51:19.505386 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:19.505430 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.65:44855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 08:51:19.505501 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9047
W20260809 08:51:19.511983 8761 catalog_manager.cc:4584] TS 93b2cb63715b45f98b39e480ba3c6d59 (127.10.184.65:44855): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 920184f82afe4716b0e34659871b9275: Network error: Client connection negotiation failed: client connection to 127.10.184.65:44855: connect: Connection refused (error 111)
I20260809 08:51:19.524449 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9225
I20260809 08:51:19.549278 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 8740
2026-08-09T08:51:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (735 ms)
[----------] 6 tests from EncryptionPolicyTest (8507 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:19 dist-test-slave-ncp9 krb5kdc[9461](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:19 dist-test-slave-ncp9 krb5kdc[9461](info): set up 2 sockets
Aug 09 08:51:19 dist-test-slave-ncp9 krb5kdc[9461](info): commencing operation
krb5kdc: starting...
W20260809 08:51:20.671802 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:20.783515 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:20.861894 9481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:20.862082 9481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:20.862102 9481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:20.863173 9481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:20.863206 9481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:20.863219 9481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:20.863230 9481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:20.863245 9481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:20.864636 9481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46713
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:20.864812 9481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:20.864991 9481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:20.869588 9492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:20.869638 9489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:20.869596 9496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:20.869701 9481 server_base.cc:1034] running on GCE node
I20260809 08:51:20.869935 9481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:20.870153 9481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:20.871282 9481 hybrid_clock.cc:648] HybridClock initialized: now 1786265480871271 us; error 26 us; skew 500 ppm
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9461](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265480, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:20.873548 9481 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:20.873975 9481 webserver.cc:460] Webserver started at http://127.0.0.1:40623/ using document root <none> and password file <none>
I20260809 08:51:20.874176 9481 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:20.874217 9481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:20.874305 9481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:20.875204 9481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "0995fec4ae7b43c9ab4584596e2836a9"
format_stamp: "Formatted at 2026-08-09 08:51:20 on dist-test-slave-ncp9"
server_key: "a3607570b0a051f531a3585d5629250d"
server_key_iv: "3036efab8f31d0b11c68c12b8c4e8794"
server_key_version: "encryptionkey@0"
I20260809 08:51:20.875486 9481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "0995fec4ae7b43c9ab4584596e2836a9"
format_stamp: "Formatted at 2026-08-09 08:51:20 on dist-test-slave-ncp9"
server_key: "a3607570b0a051f531a3585d5629250d"
server_key_iv: "3036efab8f31d0b11c68c12b8c4e8794"
server_key_version: "encryptionkey@0"
I20260809 08:51:20.876598 9481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:20.877261 9506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:20.877410 9481 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:20.877488 9481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "0995fec4ae7b43c9ab4584596e2836a9"
format_stamp: "Formatted at 2026-08-09 08:51:20 on dist-test-slave-ncp9"
server_key: "a3607570b0a051f531a3585d5629250d"
server_key_iv: "3036efab8f31d0b11c68c12b8c4e8794"
server_key_version: "encryptionkey@0"
I20260809 08:51:20.877540 9481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:20.894905 9481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:20.896423 9481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:20.896525 9481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:20.900166 9481 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46713
I20260809 08:51:20.900199 9587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46713 every 8 connection(s)
I20260809 08:51:20.900492 9481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:20.901345 9589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:20.903505 9589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9: Bootstrap starting.
I20260809 08:51:20.904048 9589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:20.904305 9589 log.cc:824] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:20.904933 9589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9: No bootstrap required, opened a new log
I20260809 08:51:20.906440 9589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260809 08:51:20.906572 9589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:20.906594 9589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0995fec4ae7b43c9ab4584596e2836a9, State: Initialized, Role: FOLLOWER
I20260809 08:51:20.906742 9589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260809 08:51:20.906816 9589 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:20.906840 9589 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:20.906883 9589 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:20.907364 9589 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260809 08:51:20.907454 9589 leader_election.cc:302] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0995fec4ae7b43c9ab4584596e2836a9; no voters:
I20260809 08:51:20.907617 9589 leader_election.cc:288] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:20.907639 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9481
I20260809 08:51:20.907642 9594 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:20.907742 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:20.907763 9594 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [term 1 LEADER]: Becoming Leader. State: Replica: 0995fec4ae7b43c9ab4584596e2836a9, State: Running, Role: LEADER
I20260809 08:51:20.907835 9594 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } }
I20260809 08:51:20.907881 9589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:20.907939 10977 external_mini_cluster.cc:1428] Setting key 894a5f5a9a8a7bdf1b8972777c030f27
I20260809 08:51:20.908139 9594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0995fec4ae7b43c9ab4584596e2836a9. Latest consensus state: current_term: 1 leader_uuid: "0995fec4ae7b43c9ab4584596e2836a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260809 08:51:20.908210 9594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:20.908403 9596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0995fec4ae7b43c9ab4584596e2836a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0995fec4ae7b43c9ab4584596e2836a9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46713 } } }
I20260809 08:51:20.908488 9596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:20.909260 9598 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:20.909788 9598 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:20.911270 9598 catalog_manager.cc:1347] Generated new cluster ID: e12ebd0f2ee54baa916591858a9ba371
I20260809 08:51:20.911324 9598 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9461](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:51:20.918258 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:20.908507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49133 (local address 127.0.0.1:46713)
0809 08:51:20.908678 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:20.908689 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:20.908723 (+ 34us) server_negotiation.cc:407] Connection header received
0809 08:51:20.908846 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:20.908852 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:20.908929 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:20.909700 (+ 771us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:20.909964 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:20.911279 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:20.912044 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:20.912431 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:20.912525 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:20.914783 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:20.914788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:20.914788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:20.914800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:20.916710 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:20.917111 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:20.917114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:20.917114 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:20.917215 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:20.917420 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:20.917421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:20.917421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:20.917639 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:20.917789 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:20.917998 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 08:51:20.918095 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260809 08:51:20.919188 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:20.926326 9605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:20.920435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42833 (local address 127.0.0.1:46713)
0809 08:51:20.920470 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:20.920472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:20.921067 (+ 595us) server_negotiation.cc:407] Connection header received
0809 08:51:20.921119 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:20.921120 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:20.921130 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:20.921188 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:20.921645 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:20.922091 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:20.922756 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:20.922976 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:20.923028 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:20.924092 (+ 1064us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:20.924094 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:20.924094 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:20.924101 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:20.925307 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:20.925629 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:20.925630 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:20.925630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:20.925679 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:20.925862 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:20.925862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:20.925863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:20.925959 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:20.926043 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:20.926191 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:51:20.926246 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:51:20.933568 9598 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:20.934067 9598 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:20.936501 9598 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0995fec4ae7b43c9ab4584596e2836a9: Generated new TSK 0
I20260809 08:51:20.936658 9598 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:20.942420 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9481
W20260809 08:51:20.954898 9458 connection.cc:535] client connection to 127.0.0.1:46713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46713): Connection reset by peer (error 104)
2026-08-09T08:51:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9631](info): set up 2 sockets
Aug 09 08:51:20 dist-test-slave-ncp9 krb5kdc[9631](info): commencing operation
krb5kdc: starting...
W20260809 08:51:22.072113 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:22 dist-test-slave-ncp9 krb5kdc[9631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:22.183337 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:22.262126 9651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:22.262343 9651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:22.262374 9651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:22.263473 9651 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:22.263512 9651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:22.263531 9651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:22.263547 9651 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:22.263561 9651 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:22.265051 9651 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:22.265290 9651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:22.265493 9651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:22.270180 9657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:22.270349 9651 server_base.cc:1034] running on GCE node
W20260809 08:51:22.270182 9663 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:22.270205 9660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:22.270635 9651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:22.270913 9651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:22.272049 9651 hybrid_clock.cc:648] HybridClock initialized: now 1786265482272038 us; error 29 us; skew 500 ppm
Aug 09 08:51:22 dist-test-slave-ncp9 krb5kdc[9631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265482, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:22.277801 9651 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:22.278214 9651 webserver.cc:460] Webserver started at http://127.0.0.1:42157/ using document root <none> and password file <none>
I20260809 08:51:22.278420 9651 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:22.278466 9651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:22.278586 9651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:22.279556 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "c5045333066448c98a1083cb39d117c8"
format_stamp: "Formatted at 2026-08-09 08:51:22 on dist-test-slave-ncp9"
server_key: "dd77ae0f60395c3a5dc5e9a77135e514"
server_key_iv: "1bb9327842013124ad66c3ef8c7c5f92"
server_key_version: "encryptionkey@0"
I20260809 08:51:22.279871 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "c5045333066448c98a1083cb39d117c8"
format_stamp: "Formatted at 2026-08-09 08:51:22 on dist-test-slave-ncp9"
server_key: "dd77ae0f60395c3a5dc5e9a77135e514"
server_key_iv: "1bb9327842013124ad66c3ef8c7c5f92"
server_key_version: "encryptionkey@0"
I20260809 08:51:22.281169 9651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:22.281795 9679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:22.281939 9651 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:22.282021 9651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "c5045333066448c98a1083cb39d117c8"
format_stamp: "Formatted at 2026-08-09 08:51:22 on dist-test-slave-ncp9"
server_key: "dd77ae0f60395c3a5dc5e9a77135e514"
server_key_iv: "1bb9327842013124ad66c3ef8c7c5f92"
server_key_version: "encryptionkey@0"
I20260809 08:51:22.282078 9651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:22.291801 9651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:22.293382 9651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:22.293505 9651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:22.297111 9651 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33037
I20260809 08:51:22.297142 9749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33037 every 8 connection(s)
I20260809 08:51:22.297441 9651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:22.298276 9750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:22.300412 9750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8: Bootstrap starting.
I20260809 08:51:22.300998 9750 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:22.301261 9750 log.cc:824] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:22.301862 9750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8: No bootstrap required, opened a new log
I20260809 08:51:22.303395 9750 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } }
I20260809 08:51:22.303525 9750 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:22.303557 9750 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5045333066448c98a1083cb39d117c8, State: Initialized, Role: FOLLOWER
I20260809 08:51:22.303687 9750 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } }
I20260809 08:51:22.303762 9750 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:22.303797 9750 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:22.303840 9750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:22.304337 9750 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } }
I20260809 08:51:22.304435 9750 leader_election.cc:302] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5045333066448c98a1083cb39d117c8; no voters:
I20260809 08:51:22.304589 9750 leader_election.cc:288] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:22.304620 9753 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:22.304782 9753 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [term 1 LEADER]: Becoming Leader. State: Replica: c5045333066448c98a1083cb39d117c8, State: Running, Role: LEADER
I20260809 08:51:22.304823 9750 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:22.304872 9753 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } }
I20260809 08:51:22.305238 9757 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5045333066448c98a1083cb39d117c8. Latest consensus state: current_term: 1 leader_uuid: "c5045333066448c98a1083cb39d117c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } } }
I20260809 08:51:22.305320 9757 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:22.305536 9754 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5045333066448c98a1083cb39d117c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5045333066448c98a1083cb39d117c8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33037 } } }
I20260809 08:51:22.305660 9754 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:22.305735 9765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:22.306203 9765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:22.307459 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9651
I20260809 08:51:22.307561 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:22.307583 9765 catalog_manager.cc:1347] Generated new cluster ID: 89a2ed6bcc984efc8248087a8651e727
I20260809 08:51:22.307627 9765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:22.307741 10977 external_mini_cluster.cc:1428] Setting key f75d84254a13761077efc38d5b1fcf3e
I20260809 08:51:22.313136 9765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:22.313367 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:22.308118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59783 (local address 127.0.0.1:33037)
0809 08:51:22.309064 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:22.309071 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:22.309245 (+ 174us) server_negotiation.cc:407] Connection header received
0809 08:51:22.309361 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:22.309367 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:22.309408 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:22.309516 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:22.309721 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:22.310623 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:22.311403 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:22.311810 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:22.311912 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:22.312538 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:22.312542 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:22.312542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:22.312764 (+ 222us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:22.312805 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:22.312822 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:22.313062 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:22.313108 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:51:22.313137 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":910,"thread_start_us":63,"threads_started":1}
I20260809 08:51:22.313786 9765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:22.314185 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:22.319058 9783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:22.315755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51109 (local address 127.0.0.1:33037)
0809 08:51:22.315786 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:22.315788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:22.315813 (+ 25us) server_negotiation.cc:407] Connection header received
0809 08:51:22.315830 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:22.315831 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:22.315838 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:22.315874 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:22.316000 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:22.316743 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:22.317798 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:22.318115 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:22.318172 (+ 57us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:22.318800 (+ 628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:22.318801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:22.318802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:22.318806 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:22.318819 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:22.318824 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:22.318854 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:22.318976 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 08:51:22.318988 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:51:22.321167 9765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5045333066448c98a1083cb39d117c8: Generated new TSK 0
I20260809 08:51:22.321324 9765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:22.326488 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9651
W20260809 08:51:22.342900 9629 connection.cc:535] client connection to 127.0.0.1:33037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33037): Connection reset by peer (error 104)
2026-08-09T08:51:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1387 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:22 dist-test-slave-ncp9 krb5kdc[9796](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:22 dist-test-slave-ncp9 krb5kdc[9796](info): set up 2 sockets
Aug 09 08:51:22 dist-test-slave-ncp9 krb5kdc[9796](info): commencing operation
krb5kdc: starting...
W20260809 08:51:23.453852 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:23 dist-test-slave-ncp9 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:23.567291 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:23.645712 9810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:23.645913 9810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:23.645934 9810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:23.647011 9810 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:23.647044 9810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:23.647058 9810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:23.647068 9810 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:23.647082 9810 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:23.648497 9810 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37801
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9810
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:23.648689 9810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:23.648859 9810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:23.653594 9810 server_base.cc:1034] running on GCE node
W20260809 08:51:23.653559 9815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:23.653554 9816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:23.653559 9818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:23.653990 9810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:23.654259 9810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:23.655401 9810 hybrid_clock.cc:648] HybridClock initialized: now 1786265483655386 us; error 26 us; skew 500 ppm
Aug 09 08:51:23 dist-test-slave-ncp9 krb5kdc[9796](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265483, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:23.657816 9810 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:23.658217 9810 webserver.cc:460] Webserver started at http://127.0.0.1:33675/ using document root <none> and password file <none>
I20260809 08:51:23.658406 9810 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:23.658453 9810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:23.658574 9810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:23.659569 9810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "2e47b17cf47f462d83abc8c59a0ccae2"
format_stamp: "Formatted at 2026-08-09 08:51:23 on dist-test-slave-ncp9"
server_key: "0e410c95208815a2f4dd2090b672395c"
server_key_iv: "fa53b81c0ff99d43d0de2d17e5fb81fa"
server_key_version: "encryptionkey@0"
I20260809 08:51:23.659909 9810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "2e47b17cf47f462d83abc8c59a0ccae2"
format_stamp: "Formatted at 2026-08-09 08:51:23 on dist-test-slave-ncp9"
server_key: "0e410c95208815a2f4dd2090b672395c"
server_key_iv: "fa53b81c0ff99d43d0de2d17e5fb81fa"
server_key_version: "encryptionkey@0"
I20260809 08:51:23.661113 9810 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:23.661749 9831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:23.661892 9810 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:23.661978 9810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "2e47b17cf47f462d83abc8c59a0ccae2"
format_stamp: "Formatted at 2026-08-09 08:51:23 on dist-test-slave-ncp9"
server_key: "0e410c95208815a2f4dd2090b672395c"
server_key_iv: "fa53b81c0ff99d43d0de2d17e5fb81fa"
server_key_version: "encryptionkey@0"
I20260809 08:51:23.662034 9810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:23.665164 9810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:23.666759 9810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:23.666882 9810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:23.670457 9810 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37801
I20260809 08:51:23.670504 9906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37801 every 8 connection(s)
I20260809 08:51:23.670775 9810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:23.671048 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9810
I20260809 08:51:23.671180 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:23.671378 10977 external_mini_cluster.cc:1428] Setting key 246b26bf0aa23f88def70aba9c581376
I20260809 08:51:23.671744 9907 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:23.673571 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:23.671853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58297 (local address 127.0.0.1:37801)
0809 08:51:23.672021 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:23.672030 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:23.672054 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:51:23.672444 (+ 390us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:23.672450 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:23.672474 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:23.672576 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:23.673193 (+ 617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:23.673197 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:23.673197 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:23.673200 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:23.673237 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:23.673247 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:23.673297 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:23.673370 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:51:23.673401 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260809 08:51:23.674402 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:23.674933 9907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2: Bootstrap starting.
I20260809 08:51:23.675659 9907 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:23.675976 9907 log.cc:824] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:23.676726 9907 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2: No bootstrap required, opened a new log
I20260809 08:51:23.676815 9911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:23.675957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51185 (local address 127.0.0.1:37801)
0809 08:51:23.675995 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:23.675996 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:23.676047 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:51:23.676107 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:23.676108 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:23.676112 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:23.676170 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:23.676637 (+ 467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:23.676638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:23.676639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:23.676642 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:23.676655 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:23.676658 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:23.676698 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:23.676752 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 08:51:23.676761 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:23.678551 9907 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260809 08:51:23.678679 9907 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:23.678702 9907 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e47b17cf47f462d83abc8c59a0ccae2, State: Initialized, Role: FOLLOWER
I20260809 08:51:23.678790 9907 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260809 08:51:23.678874 9907 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:23.678900 9907 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:23.678932 9907 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:23.679394 9907 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260809 08:51:23.679479 9907 leader_election.cc:302] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e47b17cf47f462d83abc8c59a0ccae2; no voters:
I20260809 08:51:23.679630 9907 leader_election.cc:288] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:23.679661 9918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:23.679849 9918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [term 1 LEADER]: Becoming Leader. State: Replica: 2e47b17cf47f462d83abc8c59a0ccae2, State: Running, Role: LEADER
I20260809 08:51:23.679889 9907 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:23.679936 9918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } }
I20260809 08:51:23.680291 9918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e47b17cf47f462d83abc8c59a0ccae2. Latest consensus state: current_term: 1 leader_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } } }
I20260809 08:51:23.680433 9918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:23.680318 9923 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e47b17cf47f462d83abc8c59a0ccae2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37801 } } }
I20260809 08:51:23.680616 9923 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:23.680929 9936 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:23.681423 9936 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:23.682859 9936 catalog_manager.cc:1347] Generated new cluster ID: 6229edcdfc834791a2e0a89395458801
I20260809 08:51:23.682900 9936 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:23.687475 9936 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:23.687605 9936 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:23.692101 9936 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e47b17cf47f462d83abc8c59a0ccae2: Generated new TSK 0
I20260809 08:51:23.692251 9936 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:23.700353 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9810
2026-08-09T08:51:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1374 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:23 dist-test-slave-ncp9 krb5kdc[9952](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:23 dist-test-slave-ncp9 krb5kdc[9952](info): set up 2 sockets
Aug 09 08:51:23 dist-test-slave-ncp9 krb5kdc[9952](info): commencing operation
krb5kdc: starting...
W20260809 08:51:24.845731 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.115s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:24 dist-test-slave-ncp9 krb5kdc[9952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:24.955428 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39281
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39281
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:25.033236 9969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:25.033425 9969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:25.033447 9969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:25.034529 9969 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:25.034571 9969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:25.034584 9969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:25.034595 9969 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:25.034608 9969 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:25.036008 9969 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:25.036190 9969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:25.036360 9969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:25.040938 9979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:25.041026 9975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:25.041128 9969 server_base.cc:1034] running on GCE node
W20260809 08:51:25.040920 9974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:25.041374 9969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:25.041646 9969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:25.042784 9969 hybrid_clock.cc:648] HybridClock initialized: now 1786265485042769 us; error 29 us; skew 500 ppm
Aug 09 08:51:25 dist-test-slave-ncp9 krb5kdc[9952](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265485, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:25.045106 9969 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:25.045516 9969 webserver.cc:460] Webserver started at http://127.0.0.1:40617/ using document root <none> and password file <none>
I20260809 08:51:25.045728 9969 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:25.045775 9969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:25.045895 9969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:25.046850 9969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "ac6f35ed0e6d4c758e041e7b98408d11"
format_stamp: "Formatted at 2026-08-09 08:51:25 on dist-test-slave-ncp9"
server_key: "2a3729bb062b5b08fcce0765dd930c30"
server_key_iv: "4e29ea4b7290f95f9de5bb87aaa098f9"
server_key_version: "encryptionkey@0"
I20260809 08:51:25.047169 9969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "ac6f35ed0e6d4c758e041e7b98408d11"
format_stamp: "Formatted at 2026-08-09 08:51:25 on dist-test-slave-ncp9"
server_key: "2a3729bb062b5b08fcce0765dd930c30"
server_key_iv: "4e29ea4b7290f95f9de5bb87aaa098f9"
server_key_version: "encryptionkey@0"
I20260809 08:51:25.048413 9969 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:25.049106 9994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:25.049240 9969 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:25.049324 9969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "ac6f35ed0e6d4c758e041e7b98408d11"
format_stamp: "Formatted at 2026-08-09 08:51:25 on dist-test-slave-ncp9"
server_key: "2a3729bb062b5b08fcce0765dd930c30"
server_key_iv: "4e29ea4b7290f95f9de5bb87aaa098f9"
server_key_version: "encryptionkey@0"
I20260809 08:51:25.049381 9969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:25.064392 9969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:25.066479 9969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:25.066617 9969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:25.070314 9969 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39281
I20260809 08:51:25.070349 10068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39281 every 8 connection(s)
I20260809 08:51:25.070652 9969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:25.071452 10069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:25.073648 10069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11: Bootstrap starting.
I20260809 08:51:25.074198 10069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:25.074443 10069 log.cc:824] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:25.075060 10069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11: No bootstrap required, opened a new log
I20260809 08:51:25.076556 10069 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } }
I20260809 08:51:25.076689 10069 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:25.076710 10069 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac6f35ed0e6d4c758e041e7b98408d11, State: Initialized, Role: FOLLOWER
I20260809 08:51:25.076841 10069 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } }
I20260809 08:51:25.076949 10069 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:25.076990 10069 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:25.077039 10069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:25.077612 10069 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } }
I20260809 08:51:25.077703 10069 leader_election.cc:302] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac6f35ed0e6d4c758e041e7b98408d11; no voters:
I20260809 08:51:25.077848 10069 leader_election.cc:288] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:25.077901 10072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:25.078070 10069 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:25.078070 10072 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [term 1 LEADER]: Becoming Leader. State: Replica: ac6f35ed0e6d4c758e041e7b98408d11, State: Running, Role: LEADER
I20260809 08:51:25.078258 10072 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } }
I20260809 08:51:25.078598 10075 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac6f35ed0e6d4c758e041e7b98408d11. Latest consensus state: current_term: 1 leader_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } } }
I20260809 08:51:25.078751 10075 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:25.078603 10073 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac6f35ed0e6d4c758e041e7b98408d11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39281 } } }
I20260809 08:51:25.078961 10073 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:25.079264 10077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:25.079296 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 9969
I20260809 08:51:25.079419 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:25.079622 10977 external_mini_cluster.cc:1428] Setting key 001d03912c017122d6e42d4ff7b9261a
I20260809 08:51:25.079751 10077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:25.081341 10077 catalog_manager.cc:1347] Generated new cluster ID: 7fa6165e26c64dff8e8216de2e397e91
I20260809 08:51:25.081423 10077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:51:25 dist-test-slave-ncp9 krb5kdc[9952](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265484, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:51:25.088810 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:25.080189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46503 (local address 127.0.0.1:39281)
0809 08:51:25.080570 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:25.080578 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:25.080599 (+ 21us) server_negotiation.cc:407] Connection header received
0809 08:51:25.080688 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:25.080693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:25.080731 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:25.080836 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:25.081617 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:25.082537 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:25.083270 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:25.083690 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:25.083780 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:25.085933 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:25.085937 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:25.085938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:25.085949 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:25.087389 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:25.087709 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:25.087712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:25.087712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:25.087803 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:25.087977 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:25.087978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:25.087978 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:25.088222 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:25.088363 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:25.088481 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:51:25.088616 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":82,"threads_started":1}
I20260809 08:51:25.089643 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:25.096341 10103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:25.090914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60943 (local address 127.0.0.1:39281)
0809 08:51:25.090947 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:25.090949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:25.091109 (+ 160us) server_negotiation.cc:407] Connection header received
0809 08:51:25.091160 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:25.091168 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:25.091177 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:25.091229 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:25.091729 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:25.092220 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:25.092875 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:25.093146 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:25.093214 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:25.094218 (+ 1004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:25.094220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:25.094220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:25.094227 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:25.095401 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:25.095692 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:25.095694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:25.095694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:25.095746 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:25.095909 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:25.095910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:25.095910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:25.096010 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:25.096090 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:25.096196 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 08:51:25.096262 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 08:51:25.104110 10077 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:25.104589 10077 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:25.114042 10077 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac6f35ed0e6d4c758e041e7b98408d11: Generated new TSK 0
I20260809 08:51:25.114210 10077 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:25.118978 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 9969
2026-08-09T08:51:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:25 dist-test-slave-ncp9 krb5kdc[10117](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:25 dist-test-slave-ncp9 krb5kdc[10117](info): set up 2 sockets
Aug 09 08:51:25 dist-test-slave-ncp9 krb5kdc[10117](info): commencing operation
krb5kdc: starting...
W20260809 08:51:26.246773 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:26 dist-test-slave-ncp9 krb5kdc[10117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:26.355406 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:26.434176 10134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:26.434371 10134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:26.434404 10134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:26.435518 10134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:26.435559 10134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:26.435578 10134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:26.435595 10134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:26.435611 10134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:26.437075 10134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:26.437321 10134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:26.437517 10134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:26.442188 10145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:26.442247 10149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:26.442204 10144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:26.442235 10134 server_base.cc:1034] running on GCE node
I20260809 08:51:26.442659 10134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:26.442939 10134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:26.444079 10134 hybrid_clock.cc:648] HybridClock initialized: now 1786265486444065 us; error 28 us; skew 500 ppm
Aug 09 08:51:26 dist-test-slave-ncp9 krb5kdc[10117](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265486, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:26.446436 10134 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:26.446869 10134 webserver.cc:460] Webserver started at http://127.0.0.1:38685/ using document root <none> and password file <none>
I20260809 08:51:26.447062 10134 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:26.447108 10134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:26.447234 10134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:26.448297 10134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "1f12d815574344c68d4db2ea31ae5757"
format_stamp: "Formatted at 2026-08-09 08:51:26 on dist-test-slave-ncp9"
server_key: "a320d272d550ae587fb2e7360c7bdd31"
server_key_iv: "dc1b46485fcc00daf0b8b6e973bb0f2f"
server_key_version: "encryptionkey@0"
I20260809 08:51:26.448628 10134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "1f12d815574344c68d4db2ea31ae5757"
format_stamp: "Formatted at 2026-08-09 08:51:26 on dist-test-slave-ncp9"
server_key: "a320d272d550ae587fb2e7360c7bdd31"
server_key_iv: "dc1b46485fcc00daf0b8b6e973bb0f2f"
server_key_version: "encryptionkey@0"
I20260809 08:51:26.449965 10134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:26.450614 10158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:26.450766 10134 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:26.450850 10134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "1f12d815574344c68d4db2ea31ae5757"
format_stamp: "Formatted at 2026-08-09 08:51:26 on dist-test-slave-ncp9"
server_key: "a320d272d550ae587fb2e7360c7bdd31"
server_key_iv: "dc1b46485fcc00daf0b8b6e973bb0f2f"
server_key_version: "encryptionkey@0"
I20260809 08:51:26.450917 10134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:26.466867 10134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:26.468415 10134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:26.468537 10134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:26.472213 10134 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33363
I20260809 08:51:26.472244 10235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33363 every 8 connection(s)
I20260809 08:51:26.472538 10134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:26.473376 10236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:26.475543 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757: Bootstrap starting.
I20260809 08:51:26.476130 10236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:26.476389 10236 log.cc:824] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:26.477061 10236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757: No bootstrap required, opened a new log
I20260809 08:51:26.478852 10236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } }
I20260809 08:51:26.478986 10236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:26.479020 10236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f12d815574344c68d4db2ea31ae5757, State: Initialized, Role: FOLLOWER
I20260809 08:51:26.479225 10236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } }
I20260809 08:51:26.479274 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10134
I20260809 08:51:26.479351 10236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:26.479396 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:26.479403 10236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:26.479449 10236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:26.479589 10977 external_mini_cluster.cc:1428] Setting key 890af858ff7a84725598cd1c2651f71b
I20260809 08:51:26.480075 10236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } }
I20260809 08:51:26.480195 10236 leader_election.cc:302] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f12d815574344c68d4db2ea31ae5757; no voters:
I20260809 08:51:26.480340 10236 leader_election.cc:288] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:26.480365 10244 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:26.480479 10244 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [term 1 LEADER]: Becoming Leader. State: Replica: 1f12d815574344c68d4db2ea31ae5757, State: Running, Role: LEADER
I20260809 08:51:26.480590 10236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:26.480607 10244 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } }
I20260809 08:51:26.480960 10244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f12d815574344c68d4db2ea31ae5757. Latest consensus state: current_term: 1 leader_uuid: "1f12d815574344c68d4db2ea31ae5757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } } }
I20260809 08:51:26.481040 10244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:26.481002 10245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f12d815574344c68d4db2ea31ae5757" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f12d815574344c68d4db2ea31ae5757" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33363 } } }
I20260809 08:51:26.481086 10245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:26.482594 10250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:26.483106 10250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:26.484836 10250 catalog_manager.cc:1347] Generated new cluster ID: 026d0a14f2b54baeb8c0208ba3c436a4
I20260809 08:51:26.484905 10250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:26.486316 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:26.480067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44155 (local address 127.0.0.1:33363)
0809 08:51:26.480301 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:26.480312 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:26.481326 (+ 1014us) server_negotiation.cc:407] Connection header received
0809 08:51:26.481454 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:26.481463 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:26.481523 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:26.481689 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:26.481915 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:26.483138 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:26.484221 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:26.484659 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:26.484745 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:26.485507 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:26.485511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:26.485511 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:26.485729 (+ 218us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:26.485767 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:26.485782 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:26.485840 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:26.486051 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 08:51:26.486115 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":74,"threads_started":1}
I20260809 08:51:26.487144 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:26.492063 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:26.488570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49403 (local address 127.0.0.1:33363)
0809 08:51:26.488602 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:26.488604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:26.488722 (+ 118us) server_negotiation.cc:407] Connection header received
0809 08:51:26.488811 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:26.488812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:26.488822 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:26.488851 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:26.489069 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:26.489788 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:26.490816 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:26.491115 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:26.491155 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:26.491822 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:26.491823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:26.491824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:26.491829 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:26.491841 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:26.491845 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:26.491873 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:26.491970 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 08:51:26.491993 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:26.500324 10250 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:26.500792 10250 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:26.506681 10250 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f12d815574344c68d4db2ea31ae5757: Generated new TSK 0
I20260809 08:51:26.506841 10250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:26.511936 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10134
2026-08-09T08:51:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1395 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:26 dist-test-slave-ncp9 krb5kdc[10283](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:26 dist-test-slave-ncp9 krb5kdc[10283](info): set up 2 sockets
Aug 09 08:51:26 dist-test-slave-ncp9 krb5kdc[10283](info): commencing operation
krb5kdc: starting...
W20260809 08:51:27.647938 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:27 dist-test-slave-ncp9 krb5kdc[10283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265487, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:27.757350 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44649
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44649
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:27.836164 10301 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:27.836357 10301 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:27.836378 10301 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:27.837502 10301 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:27.837546 10301 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:27.837560 10301 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:27.837571 10301 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:27.837584 10301 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:27.838996 10301 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10301
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:27.839186 10301 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:27.839352 10301 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:27.843909 10309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:27.843917 10311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:27.843998 10313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:27.843951 10301 server_base.cc:1034] running on GCE node
I20260809 08:51:27.844296 10301 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:27.844574 10301 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:27.845705 10301 hybrid_clock.cc:648] HybridClock initialized: now 1786265487845693 us; error 29 us; skew 500 ppm
Aug 09 08:51:27 dist-test-slave-ncp9 krb5kdc[10283](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265487, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:27.851147 10301 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:27.851536 10301 webserver.cc:460] Webserver started at http://127.0.0.1:33601/ using document root <none> and password file <none>
I20260809 08:51:27.851701 10301 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:27.851738 10301 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:27.851826 10301 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:27.852782 10301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "9b0e4de43565473eb0416cc9fe186c23"
format_stamp: "Formatted at 2026-08-09 08:51:27 on dist-test-slave-ncp9"
server_key: "4de1f1d6459d1966663c21921ee902ec"
server_key_iv: "dca652cae43bf3e17d9cfa53e304e6b1"
server_key_version: "encryptionkey@0"
I20260809 08:51:27.853125 10301 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "9b0e4de43565473eb0416cc9fe186c23"
format_stamp: "Formatted at 2026-08-09 08:51:27 on dist-test-slave-ncp9"
server_key: "4de1f1d6459d1966663c21921ee902ec"
server_key_iv: "dca652cae43bf3e17d9cfa53e304e6b1"
server_key_version: "encryptionkey@0"
I20260809 08:51:27.854313 10301 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:27.854921 10320 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:27.855058 10301 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:27.855141 10301 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "9b0e4de43565473eb0416cc9fe186c23"
format_stamp: "Formatted at 2026-08-09 08:51:27 on dist-test-slave-ncp9"
server_key: "4de1f1d6459d1966663c21921ee902ec"
server_key_iv: "dca652cae43bf3e17d9cfa53e304e6b1"
server_key_version: "encryptionkey@0"
I20260809 08:51:27.855202 10301 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:27.858414 10301 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:27.860034 10301 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:27.860157 10301 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:27.863840 10301 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44649
I20260809 08:51:27.863878 10402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44649 every 8 connection(s)
I20260809 08:51:27.864154 10301 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:27.865088 10405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:27.867208 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23: Bootstrap starting.
I20260809 08:51:27.867748 10405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:27.867990 10405 log.cc:824] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:27.868607 10405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23: No bootstrap required, opened a new log
I20260809 08:51:27.870178 10405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } }
I20260809 08:51:27.870308 10405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:27.870338 10405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b0e4de43565473eb0416cc9fe186c23, State: Initialized, Role: FOLLOWER
I20260809 08:51:27.870466 10405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } }
I20260809 08:51:27.870522 10405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:27.870553 10405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:27.870599 10405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:27.871080 10405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } }
I20260809 08:51:27.871184 10405 leader_election.cc:302] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b0e4de43565473eb0416cc9fe186c23; no voters:
I20260809 08:51:27.871198 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10301
I20260809 08:51:27.871309 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:27.871337 10405 leader_election.cc:288] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:27.871377 10411 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:27.871508 10411 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [term 1 LEADER]: Becoming Leader. State: Replica: 9b0e4de43565473eb0416cc9fe186c23, State: Running, Role: LEADER
I20260809 08:51:27.871522 10977 external_mini_cluster.cc:1428] Setting key 67cbdbfc6fb7334c4c160bb834c328c6
I20260809 08:51:27.871637 10405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:27.871650 10411 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } }
I20260809 08:51:27.872140 10412 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b0e4de43565473eb0416cc9fe186c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } } }
I20260809 08:51:27.872242 10412 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:27.872150 10413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b0e4de43565473eb0416cc9fe186c23. Latest consensus state: current_term: 1 leader_uuid: "9b0e4de43565473eb0416cc9fe186c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b0e4de43565473eb0416cc9fe186c23" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44649 } } }
I20260809 08:51:27.872424 10413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:27.872471 10421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:27.872989 10421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:27.874357 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:27.872110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58341 (local address 127.0.0.1:44649)
0809 08:51:27.872276 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:27.872285 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:27.872322 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:51:27.872729 (+ 407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:27.872737 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:27.872771 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:27.873221 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:27.873735 (+ 514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:27.873738 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:27.873738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:27.873742 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:27.873782 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:27.873796 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:27.874115 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:27.874146 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 08:51:27.874188 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260809 08:51:27.874575 10433 mvcc.cc:205] Tried to move back new op lower bound from 7316543438331371520 to 7316543438323142656. Current Snapshot: MvccSnapshot[applied={T|T < 7316543438331371520}]
I20260809 08:51:27.874959 10421 catalog_manager.cc:1347] Generated new cluster ID: ac5309fda92048109cba815dcf5a4fc9
I20260809 08:51:27.875010 10421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:27.877386 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:27.881850 10420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:27.878784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39325 (local address 127.0.0.1:44649)
0809 08:51:27.878816 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:27.878818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:27.881043 (+ 2225us) server_negotiation.cc:407] Connection header received
0809 08:51:27.881100 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:27.881101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:27.881106 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:27.881150 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:27.881651 (+ 501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:27.881652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:27.881652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:27.881656 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:27.881680 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:27.881684 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:27.881732 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:27.881777 (+ 45us) server_negotiation.cc:299] Negotiation successful
0809 08:51:27.881790 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:27.884256 10421 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:27.884395 10421 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:27.887189 10421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b0e4de43565473eb0416cc9fe186c23: Generated new TSK 0
I20260809 08:51:27.887337 10421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:27.894598 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10301
2026-08-09T08:51:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:27 dist-test-slave-ncp9 krb5kdc[10452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:27 dist-test-slave-ncp9 krb5kdc[10452](info): set up 2 sockets
Aug 09 08:51:27 dist-test-slave-ncp9 krb5kdc[10452](info): commencing operation
krb5kdc: starting...
W20260809 08:51:29.023711 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:29.135501 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46677
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46677
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:29.213740 10467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:29.213918 10467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:29.213939 10467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:29.215030 10467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:29.215070 10467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:29.215083 10467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:29.215106 10467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:29.215129 10467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:29.216545 10467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46677
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46677
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:29.216758 10467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:29.216989 10467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:29.221576 10474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:29.221596 10477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:29.221570 10475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:29.221660 10467 server_base.cc:1034] running on GCE node
I20260809 08:51:29.222076 10467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:29.222368 10467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:29.223512 10467 hybrid_clock.cc:648] HybridClock initialized: now 1786265489223501 us; error 26 us; skew 500 ppm
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265489, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:29.225790 10467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:29.226217 10467 webserver.cc:460] Webserver started at http://127.0.0.1:46211/ using document root <none> and password file <none>
I20260809 08:51:29.226423 10467 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:29.226468 10467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:29.226589 10467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:29.227550 10467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "4e30c757ebf34f979ad8bb16bd892407"
format_stamp: "Formatted at 2026-08-09 08:51:29 on dist-test-slave-ncp9"
server_key: "21a489bd97a6badca73b7ddcaa251580"
server_key_iv: "9416e925ae3186a54c3385ce7f0c0042"
server_key_version: "encryptionkey@0"
I20260809 08:51:29.227875 10467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "4e30c757ebf34f979ad8bb16bd892407"
format_stamp: "Formatted at 2026-08-09 08:51:29 on dist-test-slave-ncp9"
server_key: "21a489bd97a6badca73b7ddcaa251580"
server_key_iv: "9416e925ae3186a54c3385ce7f0c0042"
server_key_version: "encryptionkey@0"
I20260809 08:51:29.229151 10467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:29.229792 10486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:29.229935 10467 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:29.230015 10467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "4e30c757ebf34f979ad8bb16bd892407"
format_stamp: "Formatted at 2026-08-09 08:51:29 on dist-test-slave-ncp9"
server_key: "21a489bd97a6badca73b7ddcaa251580"
server_key_iv: "9416e925ae3186a54c3385ce7f0c0042"
server_key_version: "encryptionkey@0"
I20260809 08:51:29.230072 10467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:29.253185 10467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:29.254747 10467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:29.254873 10467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:29.258507 10467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46677
I20260809 08:51:29.258543 10562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46677 every 8 connection(s)
I20260809 08:51:29.258837 10467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:29.259639 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10467
I20260809 08:51:29.259656 10563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:29.259756 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:29.259927 10977 external_mini_cluster.cc:1428] Setting key 0b8ea397bd8c90f68d1157f6800f3faa
I20260809 08:51:29.262696 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407: Bootstrap starting.
I20260809 08:51:29.263312 10563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:29.263613 10563 log.cc:824] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:29.264274 10563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407: No bootstrap required, opened a new log
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10452](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265489, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:51:29.266570 10563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } }
I20260809 08:51:29.266710 10563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:29.266742 10563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e30c757ebf34f979ad8bb16bd892407, State: Initialized, Role: FOLLOWER
I20260809 08:51:29.266868 10563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } }
I20260809 08:51:29.266937 10563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:29.266958 10563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:29.267002 10563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:29.267603 10563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } }
I20260809 08:51:29.267724 10563 leader_election.cc:302] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e30c757ebf34f979ad8bb16bd892407; no voters:
I20260809 08:51:29.267887 10563 leader_election.cc:288] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:29.267915 10569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:29.268018 10569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [term 1 LEADER]: Becoming Leader. State: Replica: 4e30c757ebf34f979ad8bb16bd892407, State: Running, Role: LEADER
I20260809 08:51:29.268174 10563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:29.268152 10569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } }
I20260809 08:51:29.268608 10573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e30c757ebf34f979ad8bb16bd892407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } } }
I20260809 08:51:29.268743 10573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:29.268975 10584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:29.269444 10584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:29.270380 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e30c757ebf34f979ad8bb16bd892407. Latest consensus state: current_term: 1 leader_uuid: "4e30c757ebf34f979ad8bb16bd892407" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e30c757ebf34f979ad8bb16bd892407" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46677 } } }
I20260809 08:51:29.270459 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:29.270757 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:29.260542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:35019 (local address 127.0.0.1:46677)
0809 08:51:29.260692 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:29.260701 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:29.260724 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:51:29.261176 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:29.261183 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:29.261230 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:29.261336 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:29.262099 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:29.262988 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:29.263965 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:29.264386 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:29.264467 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:29.266825 (+ 2358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:29.266829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:29.266830 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:29.266842 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:29.268519 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:29.269021 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:29.269024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:29.269024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:29.269127 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:29.269413 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:29.269414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:29.269414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:29.269717 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:29.269861 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:29.270458 (+ 597us) server_negotiation.cc:299] Negotiation successful
0809 08:51:29.270560 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260809 08:51:29.271085 10584 catalog_manager.cc:1347] Generated new cluster ID: b2d8c1fd44194f5fb30a6ef7490628b7
I20260809 08:51:29.271138 10584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:29.272614 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:29.279575 10566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:29.273894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:57667 (local address 127.0.0.1:46677)
0809 08:51:29.273926 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:29.273927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:29.274021 (+ 94us) server_negotiation.cc:407] Connection header received
0809 08:51:29.274078 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:29.274078 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:29.274087 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:29.274130 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:29.274583 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:29.275291 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:29.275952 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:29.276189 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:29.276242 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:29.277321 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:29.277323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:29.277323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:29.277330 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:29.278531 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:29.278875 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:29.278876 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:29.278876 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:29.278929 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:29.279112 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:29.279113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:29.279114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:29.279220 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:29.279304 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:29.279436 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 08:51:29.279494 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:29.295032 10584 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:29.295492 10584 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:29.297690 10584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e30c757ebf34f979ad8bb16bd892407: Generated new TSK 0
I20260809 08:51:29.297851 10584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:29.304523 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10467
W20260809 08:51:29.317374 10449 connection.cc:535] client connection to 127.0.0.1:46677 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46677): Connection reset by peer (error 104)
2026-08-09T08:51:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10609](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10609](info): set up 2 sockets
Aug 09 08:51:29 dist-test-slave-ncp9 krb5kdc[10609](info): commencing operation
krb5kdc: starting...
W20260809 08:51:30.425184 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:30 dist-test-slave-ncp9 krb5kdc[10609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265490, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:30.539520 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44141
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44141
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:30.618480 10632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:30.618673 10632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:30.618703 10632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:30.619800 10632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:30.619840 10632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:30.619858 10632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:30.619874 10632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:30.619889 10632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:30.621336 10632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:30.621568 10632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:30.621762 10632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:30.626398 10643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:30.626482 10632 server_base.cc:1034] running on GCE node
W20260809 08:51:30.626525 10645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:30.626406 10639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:30.626879 10632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:30.627151 10632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:30.628289 10632 hybrid_clock.cc:648] HybridClock initialized: now 1786265490628275 us; error 27 us; skew 500 ppm
Aug 09 08:51:30 dist-test-slave-ncp9 krb5kdc[10609](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265490, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:30.630657 10632 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:30.631067 10632 webserver.cc:460] Webserver started at http://127.0.0.1:42211/ using document root <none> and password file <none>
I20260809 08:51:30.631270 10632 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:30.631318 10632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:30.631438 10632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:30.632382 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "994f18a19ecb423b9721de8cfbe5521b"
format_stamp: "Formatted at 2026-08-09 08:51:30 on dist-test-slave-ncp9"
server_key: "9c6bc4ac1e3d85e8ce8f0bc527cd337c"
server_key_iv: "748f22eae2c61ab2f3dd7b9634d26acf"
server_key_version: "encryptionkey@0"
I20260809 08:51:30.632707 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "994f18a19ecb423b9721de8cfbe5521b"
format_stamp: "Formatted at 2026-08-09 08:51:30 on dist-test-slave-ncp9"
server_key: "9c6bc4ac1e3d85e8ce8f0bc527cd337c"
server_key_iv: "748f22eae2c61ab2f3dd7b9634d26acf"
server_key_version: "encryptionkey@0"
I20260809 08:51:30.633967 10632 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:51:30.634594 10652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:30.634742 10632 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:30.634814 10632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "994f18a19ecb423b9721de8cfbe5521b"
format_stamp: "Formatted at 2026-08-09 08:51:30 on dist-test-slave-ncp9"
server_key: "9c6bc4ac1e3d85e8ce8f0bc527cd337c"
server_key_iv: "748f22eae2c61ab2f3dd7b9634d26acf"
server_key_version: "encryptionkey@0"
I20260809 08:51:30.634878 10632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:30.642743 10632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:30.644619 10632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:30.644752 10632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:30.648319 10632 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44141
I20260809 08:51:30.648348 10732 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44141 every 8 connection(s)
I20260809 08:51:30.648633 10632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:30.649470 10733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:30.651489 10733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b: Bootstrap starting.
I20260809 08:51:30.651997 10733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:30.652216 10733 log.cc:824] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:30.652802 10733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b: No bootstrap required, opened a new log
I20260809 08:51:30.653420 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10632
I20260809 08:51:30.653537 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:30.653729 10977 external_mini_cluster.cc:1428] Setting key b641ee863417afc2e4a521ef0de71956
I20260809 08:51:30.654742 10733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } }
I20260809 08:51:30.654883 10733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:30.654918 10733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 994f18a19ecb423b9721de8cfbe5521b, State: Initialized, Role: FOLLOWER
I20260809 08:51:30.655040 10733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } }
I20260809 08:51:30.655123 10733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:30.655153 10733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:30.655200 10733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:30.655692 10733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } }
I20260809 08:51:30.655777 10733 leader_election.cc:302] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 994f18a19ecb423b9721de8cfbe5521b; no voters:
I20260809 08:51:30.655920 10733 leader_election.cc:288] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:30.655963 10742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:30.656111 10742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [term 1 LEADER]: Becoming Leader. State: Replica: 994f18a19ecb423b9721de8cfbe5521b, State: Running, Role: LEADER
I20260809 08:51:30.656173 10733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:30.656206 10742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } }
I20260809 08:51:30.656529 10742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 994f18a19ecb423b9721de8cfbe5521b. Latest consensus state: current_term: 1 leader_uuid: "994f18a19ecb423b9721de8cfbe5521b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } } }
I20260809 08:51:30.656599 10744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "994f18a19ecb423b9721de8cfbe5521b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "994f18a19ecb423b9721de8cfbe5521b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44141 } } }
I20260809 08:51:30.656770 10744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:30.656991 10742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:30.657383 10762 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:30.657893 10762 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:30.659399 10762 catalog_manager.cc:1347] Generated new cluster ID: af4129bbe5564860942cc836ed7425e1
I20260809 08:51:30.659446 10762 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:30.659596 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:30.654216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:40825 (local address 127.0.0.1:44141)
0809 08:51:30.654373 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:30.654380 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:30.654403 (+ 23us) server_negotiation.cc:407] Connection header received
0809 08:51:30.654708 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:30.654713 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:30.654755 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:30.654852 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:30.655097 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:30.656473 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:30.657626 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:30.658198 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:30.658307 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:30.658896 (+ 589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:30.658900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:30.658900 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:30.659103 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:30.659131 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:30.659142 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:30.659217 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:30.659331 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 08:51:30.659359 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":54,"threads_started":1}
I20260809 08:51:30.660418 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:30.668747 10741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:30.665213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:36019 (local address 127.0.0.1:44141)
0809 08:51:30.665260 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:30.665262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:30.665505 (+ 243us) server_negotiation.cc:407] Connection header received
0809 08:51:30.665570 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:30.665571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:30.665580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:30.665629 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:30.665756 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:30.666431 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:30.667437 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:30.667773 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:30.667848 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:30.668501 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:30.668503 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:30.668503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:30.668507 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:30.668521 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:30.668525 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:30.668573 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:30.668644 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 08:51:30.668656 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 08:51:30.686746 10762 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:30.687239 10762 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:30.691020 10762 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 994f18a19ecb423b9721de8cfbe5521b: Generated new TSK 0
I20260809 08:51:30.691186 10762 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:30.700080 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10632
W20260809 08:51:30.711526 10604 connection.cc:535] client connection to 127.0.0.1:44141 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44141): Connection reset by peer (error 104)
2026-08-09T08:51:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:30 dist-test-slave-ncp9 krb5kdc[10787](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:30 dist-test-slave-ncp9 krb5kdc[10787](info): set up 2 sockets
Aug 09 08:51:30 dist-test-slave-ncp9 krb5kdc[10787](info): commencing operation
krb5kdc: starting...
W20260809 08:51:31.820601 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:31 dist-test-slave-ncp9 krb5kdc[10787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265491, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:31.933276 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:32.012696 10808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:32.012907 10808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:32.012939 10808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:32.014062 10808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:32.014096 10808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:32.014109 10808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:32.014120 10808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:32.014133 10808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:32.015565 10808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:32.015751 10808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:32.015924 10808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:32.020496 10816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:32.020561 10818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:32.020601 10808 server_base.cc:1034] running on GCE node
W20260809 08:51:32.020541 10815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:32.020954 10808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:32.021242 10808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:32.022392 10808 hybrid_clock.cc:648] HybridClock initialized: now 1786265492022380 us; error 29 us; skew 500 ppm
Aug 09 08:51:32 dist-test-slave-ncp9 krb5kdc[10787](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265492, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:32.024725 10808 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:32.025154 10808 webserver.cc:460] Webserver started at http://127.0.0.1:34273/ using document root <none> and password file <none>
I20260809 08:51:32.025360 10808 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:32.025398 10808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:32.025486 10808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:32.026434 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "e97859c0494745dea81ab749c07a750d"
format_stamp: "Formatted at 2026-08-09 08:51:32 on dist-test-slave-ncp9"
server_key: "8f20a58389ba81361e5ace5e8b71dc13"
server_key_iv: "725f8534bfed7a4e16395e5b40a628c0"
server_key_version: "encryptionkey@0"
I20260809 08:51:32.026715 10808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "e97859c0494745dea81ab749c07a750d"
format_stamp: "Formatted at 2026-08-09 08:51:32 on dist-test-slave-ncp9"
server_key: "8f20a58389ba81361e5ace5e8b71dc13"
server_key_iv: "725f8534bfed7a4e16395e5b40a628c0"
server_key_version: "encryptionkey@0"
I20260809 08:51:32.027961 10808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:32.028597 10829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:32.028767 10808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:32.028853 10808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "e97859c0494745dea81ab749c07a750d"
format_stamp: "Formatted at 2026-08-09 08:51:32 on dist-test-slave-ncp9"
server_key: "8f20a58389ba81361e5ace5e8b71dc13"
server_key_iv: "725f8534bfed7a4e16395e5b40a628c0"
server_key_version: "encryptionkey@0"
I20260809 08:51:32.028930 10808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:32.032078 10808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:32.033779 10808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:32.033879 10808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:32.037474 10808 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39653
I20260809 08:51:32.037509 10902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39653 every 8 connection(s)
I20260809 08:51:32.037776 10808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:32.038693 10903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:32.040793 10903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d: Bootstrap starting.
I20260809 08:51:32.041352 10903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:32.041599 10903 log.cc:824] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:32.042232 10903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d: No bootstrap required, opened a new log
I20260809 08:51:32.043759 10903 raft_consensus.cc:348] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260809 08:51:32.043887 10903 raft_consensus.cc:374] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:32.043920 10903 raft_consensus.cc:729] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e97859c0494745dea81ab749c07a750d, State: Initialized, Role: FOLLOWER
I20260809 08:51:32.044054 10903 consensus_queue.cc:260] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260809 08:51:32.044130 10903 raft_consensus.cc:388] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:32.044162 10903 raft_consensus.cc:482] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:32.044209 10903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:32.044694 10903 raft_consensus.cc:504] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260809 08:51:32.044790 10903 leader_election.cc:302] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e97859c0494745dea81ab749c07a750d; no voters:
I20260809 08:51:32.044961 10903 leader_election.cc:288] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:32.044989 10909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:32.045120 10909 raft_consensus.cc:686] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [term 1 LEADER]: Becoming Leader. State: Replica: e97859c0494745dea81ab749c07a750d, State: Running, Role: LEADER
I20260809 08:51:32.045207 10903 sys_catalog.cc:564] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:32.045248 10909 consensus_queue.cc:237] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } }
I20260809 08:51:32.045614 10912 sys_catalog.cc:455] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e97859c0494745dea81ab749c07a750d. Latest consensus state: current_term: 1 leader_uuid: "e97859c0494745dea81ab749c07a750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260809 08:51:32.045758 10912 sys_catalog.cc:458] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:32.045732 10910 sys_catalog.cc:455] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e97859c0494745dea81ab749c07a750d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e97859c0494745dea81ab749c07a750d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39653 } } }
I20260809 08:51:32.045855 10910 sys_catalog.cc:458] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:32.046236 10923 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:32.046674 10923 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:32.047555 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10808
I20260809 08:51:32.047670 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:32.047856 10977 external_mini_cluster.cc:1428] Setting key a50a8fa9a390ab1c3470e474a15bf639
I20260809 08:51:32.048048 10923 catalog_manager.cc:1347] Generated new cluster ID: dedfda2cc5a540ebbaf1f5a7c9c155a4
I20260809 08:51:32.048092 10923 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:32.054850 10923 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:32.055022 10923 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:32.056275 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:32.048290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:46973 (local address 127.0.0.1:39653)
0809 08:51:32.054913 (+ 6623us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:32.054922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:32.054958 (+ 36us) server_negotiation.cc:407] Connection header received
0809 08:51:32.055062 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:32.055068 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:32.055096 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:32.055236 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:32.055868 (+ 632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:32.055872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:32.055872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:32.055877 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:32.055914 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:32.055926 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:32.055978 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:32.056041 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 08:51:32.056085 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6576,"thread_start_us":64,"threads_started":1}
I20260809 08:51:32.057112 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:32.062191 10940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:32.061178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:60007 (local address 127.0.0.1:39653)
0809 08:51:32.061213 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:32.061215 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:32.061364 (+ 149us) server_negotiation.cc:407] Connection header received
0809 08:51:32.061420 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:32.061420 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:32.061426 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:32.061470 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:32.061992 (+ 522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:32.061993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:32.061994 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:32.061998 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:32.062012 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:32.062016 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:32.062063 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:32.062111 (+ 48us) server_negotiation.cc:299] Negotiation successful
0809 08:51:32.062125 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:51:32.071568 10923 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e97859c0494745dea81ab749c07a750d: Generated new TSK 0
I20260809 08:51:32.071748 10923 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:32.080021 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10808
2026-08-09T08:51:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:32 dist-test-slave-ncp9 krb5kdc[10953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:32 dist-test-slave-ncp9 krb5kdc[10953](info): set up 2 sockets
Aug 09 08:51:32 dist-test-slave-ncp9 krb5kdc[10953](info): commencing operation
krb5kdc: starting...
W20260809 08:51:33.239676 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.129s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[10953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:33.348937 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:33.427531 10974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:33.427732 10974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:33.427753 10974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:33.428818 10974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:33.428850 10974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:33.428864 10974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:33.428880 10974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:33.428912 10974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:33.430365 10974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42391
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:33.430550 10974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:33.430721 10974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:33.435369 10997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:33.435413 10991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:33.435417 10974 server_base.cc:1034] running on GCE node
W20260809 08:51:33.435384 10993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:33.435766 10974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:33.436039 10974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:33.437172 10974 hybrid_clock.cc:648] HybridClock initialized: now 1786265493437155 us; error 30 us; skew 500 ppm
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[10953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265493, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:33.442438 10974 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:33.442889 10974 webserver.cc:460] Webserver started at http://127.0.0.1:42115/ using document root <none> and password file <none>
I20260809 08:51:33.443100 10974 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:33.443159 10974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:33.443291 10974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:33.444252 10974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "4ead912a92934fa78b02f249e93c05b0"
format_stamp: "Formatted at 2026-08-09 08:51:33 on dist-test-slave-ncp9"
server_key: "36271415604f775876a78abc9111aab8"
server_key_iv: "2559a1750c8e0fc65c24f2d715f89312"
server_key_version: "encryptionkey@0"
I20260809 08:51:33.444581 10974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "4ead912a92934fa78b02f249e93c05b0"
format_stamp: "Formatted at 2026-08-09 08:51:33 on dist-test-slave-ncp9"
server_key: "36271415604f775876a78abc9111aab8"
server_key_iv: "2559a1750c8e0fc65c24f2d715f89312"
server_key_version: "encryptionkey@0"
I20260809 08:51:33.445847 10974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:33.446686 11004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:33.446880 10974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:51:33.446980 10974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "4ead912a92934fa78b02f249e93c05b0"
format_stamp: "Formatted at 2026-08-09 08:51:33 on dist-test-slave-ncp9"
server_key: "36271415604f775876a78abc9111aab8"
server_key_iv: "2559a1750c8e0fc65c24f2d715f89312"
server_key_version: "encryptionkey@0"
I20260809 08:51:33.447057 10974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:33.458971 10974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:33.460988 10974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:33.461117 10974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:33.464838 10974 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42391
I20260809 08:51:33.464877 11101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42391 every 8 connection(s)
I20260809 08:51:33.465211 10974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:33.466001 11102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:33.468132 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0: Bootstrap starting.
I20260809 08:51:33.468674 11102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:33.468966 11102 log.cc:824] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:33.469591 11102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0: No bootstrap required, opened a new log
I20260809 08:51:33.471092 11102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } }
I20260809 08:51:33.471222 11102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:33.471256 11102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ead912a92934fa78b02f249e93c05b0, State: Initialized, Role: FOLLOWER
I20260809 08:51:33.471390 11102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } }
I20260809 08:51:33.471448 11102 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:33.471469 11102 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:33.471511 11102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:33.472002 11102 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } }
I20260809 08:51:33.472093 11102 leader_election.cc:302] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ead912a92934fa78b02f249e93c05b0; no voters:
I20260809 08:51:33.472239 11102 leader_election.cc:288] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:33.472265 11106 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:33.472395 11106 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [term 1 LEADER]: Becoming Leader. State: Replica: 4ead912a92934fa78b02f249e93c05b0, State: Running, Role: LEADER
I20260809 08:51:33.472479 11102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:33.472489 11106 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } }
I20260809 08:51:33.472998 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 10974
I20260809 08:51:33.473008 11107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ead912a92934fa78b02f249e93c05b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } } }
I20260809 08:51:33.473083 11107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:33.473115 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:33.473272 11109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ead912a92934fa78b02f249e93c05b0. Latest consensus state: current_term: 1 leader_uuid: "4ead912a92934fa78b02f249e93c05b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ead912a92934fa78b02f249e93c05b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42391 } } }
I20260809 08:51:33.473325 10977 external_mini_cluster.cc:1428] Setting key 1c0d3e3f4a655d725c8da096bb3b8092
I20260809 08:51:33.473332 11109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:33.473917 11119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:33.474510 11119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:33.475973 11119 catalog_manager.cc:1347] Generated new cluster ID: 2d033eacdd7640c1b814fe4942b631b1
I20260809 08:51:33.476027 11119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[10953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265493, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 08:51:33.482527 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:33.473757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:48539 (local address 127.0.0.1:42391)
0809 08:51:33.473903 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:33.473913 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:33.473948 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:51:33.474061 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:33.474069 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:33.474129 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:33.474303 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:33.474947 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:33.476134 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:33.476860 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:33.477312 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:33.477407 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:33.479521 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:33.479525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:33.479526 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:33.479537 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:33.481036 (+ 1499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:33.481388 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:33.481391 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:33.481391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:33.481485 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:33.481696 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:33.481697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:33.481698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:33.481923 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:33.482069 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:33.482217 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 08:51:33.482321 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":64,"threads_started":1}
I20260809 08:51:33.482726 11119 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:33.483371 11119 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:33.483438 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:33.489216 11119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ead912a92934fa78b02f249e93c05b0: Generated new TSK 0
I20260809 08:51:33.489367 11119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:33.490414 11130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:33.484649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:60741 (local address 127.0.0.1:42391)
0809 08:51:33.484955 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:33.484956 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:33.484963 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:33.484980 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:33.484981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:33.484988 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:33.485032 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:33.485529 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:33.486013 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:33.486674 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:33.486896 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:33.486947 (+ 51us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:33.487943 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:33.487944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:33.487944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:33.487951 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:33.489374 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:33.489728 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:33.489729 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:33.489729 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:33.489793 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:33.489970 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:33.489970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:33.489971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:33.490086 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:33.490193 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:33.490266 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 08:51:33.490330 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286}
I20260809 08:51:33.492008 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 10974
W20260809 08:51:33.517853 10951 connection.cc:535] client connection to 127.0.0.1:42391 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42391): Connection reset by peer (error 104)
2026-08-09T08:51:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[11148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[11148](info): set up 2 sockets
Aug 09 08:51:33 dist-test-slave-ncp9 krb5kdc[11148](info): commencing operation
krb5kdc: starting...
W20260809 08:51:34.653920 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:34 dist-test-slave-ncp9 krb5kdc[11148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265494, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:34.765810 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40815
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40815
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:34.844090 11168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:34.844285 11168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:34.844305 11168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:34.845739 11168 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:34.845794 11168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:34.845813 11168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:34.845841 11168 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:34.845868 11168 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:34.847457 11168 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40815
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:34.847688 11168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:34.847890 11168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:34.852483 11174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:34.852537 11175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:34.852646 11168 server_base.cc:1034] running on GCE node
W20260809 08:51:34.852717 11178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:34.852984 11168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:34.853262 11168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:34.854403 11168 hybrid_clock.cc:648] HybridClock initialized: now 1786265494854389 us; error 30 us; skew 500 ppm
Aug 09 08:51:34 dist-test-slave-ncp9 krb5kdc[11148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265494, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:34.856698 11168 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:34.857102 11168 webserver.cc:460] Webserver started at http://127.0.0.1:40131/ using document root <none> and password file <none>
I20260809 08:51:34.857268 11168 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:34.857306 11168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:34.857393 11168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:34.858294 11168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "fe97ee4116c74bb7a8422509b67c27d2"
format_stamp: "Formatted at 2026-08-09 08:51:34 on dist-test-slave-ncp9"
server_key: "f93df3480b27db2ae48eb36e75d0876d"
server_key_iv: "f7ce8980d96976debdfd40d8fadd8cc2"
server_key_version: "encryptionkey@0"
I20260809 08:51:34.858577 11168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "fe97ee4116c74bb7a8422509b67c27d2"
format_stamp: "Formatted at 2026-08-09 08:51:34 on dist-test-slave-ncp9"
server_key: "f93df3480b27db2ae48eb36e75d0876d"
server_key_iv: "f7ce8980d96976debdfd40d8fadd8cc2"
server_key_version: "encryptionkey@0"
I20260809 08:51:34.859719 11168 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:34.860325 11188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:34.860487 11168 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:34.860571 11168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "fe97ee4116c74bb7a8422509b67c27d2"
format_stamp: "Formatted at 2026-08-09 08:51:34 on dist-test-slave-ncp9"
server_key: "f93df3480b27db2ae48eb36e75d0876d"
server_key_iv: "f7ce8980d96976debdfd40d8fadd8cc2"
server_key_version: "encryptionkey@0"
I20260809 08:51:34.860644 11168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:34.878734 11168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:34.880831 11168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:34.880991 11168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:34.884860 11168 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40815
I20260809 08:51:34.884917 11263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40815 every 8 connection(s)
I20260809 08:51:34.885236 11168 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:34.886044 11264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:34.888235 11264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2: Bootstrap starting.
I20260809 08:51:34.888773 11264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:34.889048 11264 log.cc:824] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:34.889706 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11168
I20260809 08:51:34.889746 11264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2: No bootstrap required, opened a new log
I20260809 08:51:34.889811 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:34.890021 10977 external_mini_cluster.cc:1428] Setting key d317d962210df100cea499445ffaad47
I20260809 08:51:34.891683 11264 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 08:51:34.891849 11264 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:34.891919 11264 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe97ee4116c74bb7a8422509b67c27d2, State: Initialized, Role: FOLLOWER
I20260809 08:51:34.892051 11264 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 08:51:34.892120 11264 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:34.892143 11264 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:34.892186 11264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:34.892695 11264 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 08:51:34.892778 11264 leader_election.cc:302] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe97ee4116c74bb7a8422509b67c27d2; no voters:
I20260809 08:51:34.892961 11264 leader_election.cc:288] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:34.892980 11271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:34.893092 11271 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [term 1 LEADER]: Becoming Leader. State: Replica: fe97ee4116c74bb7a8422509b67c27d2, State: Running, Role: LEADER
I20260809 08:51:34.893217 11264 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:34.893240 11271 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } }
I20260809 08:51:34.893570 11271 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe97ee4116c74bb7a8422509b67c27d2. Latest consensus state: current_term: 1 leader_uuid: "fe97ee4116c74bb7a8422509b67c27d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } } }
I20260809 08:51:34.893646 11271 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:34.893622 11273 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe97ee4116c74bb7a8422509b67c27d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe97ee4116c74bb7a8422509b67c27d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40815 } } }
I20260809 08:51:34.893945 11273 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:34.894668 11285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:34.895115 11285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:34.896525 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:34.890495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:33005 (local address 127.0.0.1:40815)
0809 08:51:34.890671 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:34.890680 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:34.890713 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:51:34.891643 (+ 930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:34.891652 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:34.891728 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:34.891875 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:34.892102 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:34.893554 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:34.894620 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:34.895088 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:34.895186 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:34.895813 (+ 627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:34.895817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:34.895817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:34.896004 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:34.896040 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:34.896054 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:34.896132 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:34.896318 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 08:51:34.896346 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260809 08:51:34.897461 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:34.901479 11270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:34.898669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:57157 (local address 127.0.0.1:40815)
0809 08:51:34.898702 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:34.898703 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:34.898783 (+ 80us) server_negotiation.cc:407] Connection header received
0809 08:51:34.898856 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:34.898857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:34.898864 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:34.898907 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:34.899029 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:34.899451 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:34.900443 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:34.900624 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:34.900674 (+ 50us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:34.901284 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:34.901285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:34.901286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:34.901290 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:34.901301 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:34.901305 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:34.901343 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:34.901423 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:51:34.901433 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:51:34.905251 11285 catalog_manager.cc:1347] Generated new cluster ID: 75838b9434d049779416a629d59d2708
I20260809 08:51:34.905303 11285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:34.922391 11285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:34.922873 11285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:34.926674 11285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe97ee4116c74bb7a8422509b67c27d2: Generated new TSK 0
I20260809 08:51:34.926823 11285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:34.929888 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11168
W20260809 08:51:34.946364 11144 connection.cc:535] client connection to 127.0.0.1:40815 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40815): Connection reset by peer (error 104)
2026-08-09T08:51:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:34 dist-test-slave-ncp9 krb5kdc[11313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:34 dist-test-slave-ncp9 krb5kdc[11313](info): set up 2 sockets
Aug 09 08:51:34 dist-test-slave-ncp9 krb5kdc[11313](info): commencing operation
krb5kdc: starting...
W20260809 08:51:36.049216 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.090s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:36 dist-test-slave-ncp9 krb5kdc[11313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265496, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 08:51:36.159250 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42459
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:36.238023 11328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:36.238214 11328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:36.238234 11328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:36.239282 11328 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:36.239316 11328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:36.239328 11328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:36.239339 11328 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:36.239352 11328 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:36.240751 11328 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42459
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:36.240972 11328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:36.241151 11328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:36.245813 11328 server_base.cc:1034] running on GCE node
W20260809 08:51:36.245786 11341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:36.245761 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:36.245786 11336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:36.246266 11328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:36.246539 11328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:36.247684 11328 hybrid_clock.cc:648] HybridClock initialized: now 1786265496247669 us; error 26 us; skew 500 ppm
Aug 09 08:51:36 dist-test-slave-ncp9 krb5kdc[11313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265496, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:36.249958 11328 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 08:51:36.250351 11328 webserver.cc:460] Webserver started at http://127.0.0.1:43889/ using document root <none> and password file <none>
I20260809 08:51:36.250545 11328 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:36.250589 11328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:36.250710 11328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:36.251653 11328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "70bdffd49c934159a22598b8d3bf6113"
format_stamp: "Formatted at 2026-08-09 08:51:36 on dist-test-slave-ncp9"
server_key: "6639fee02503e01663ffe8617bd4617f"
server_key_iv: "a6dbf44531408315a80e1628cb30b609"
server_key_version: "encryptionkey@0"
I20260809 08:51:36.251972 11328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "70bdffd49c934159a22598b8d3bf6113"
format_stamp: "Formatted at 2026-08-09 08:51:36 on dist-test-slave-ncp9"
server_key: "6639fee02503e01663ffe8617bd4617f"
server_key_iv: "a6dbf44531408315a80e1628cb30b609"
server_key_version: "encryptionkey@0"
I20260809 08:51:36.253191 11328 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 08:51:36.253819 11353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:36.253961 11328 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:36.254043 11328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "70bdffd49c934159a22598b8d3bf6113"
format_stamp: "Formatted at 2026-08-09 08:51:36 on dist-test-slave-ncp9"
server_key: "6639fee02503e01663ffe8617bd4617f"
server_key_iv: "a6dbf44531408315a80e1628cb30b609"
server_key_version: "encryptionkey@0"
I20260809 08:51:36.254099 11328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:36.257215 11328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:36.258782 11328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:36.258903 11328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:36.262542 11328 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42459
I20260809 08:51:36.262574 11433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42459 every 8 connection(s)
I20260809 08:51:36.262845 11328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:36.263173 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11328
I20260809 08:51:36.263279 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:36.263404 10977 external_mini_cluster.cc:1428] Setting key 4c13d4ca0f29ca3c49d5c24b51fe4b55
I20260809 08:51:36.264412 11435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:36.265942 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:36.263846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:59759 (local address 127.0.0.1:42459)
0809 08:51:36.264048 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:36.264059 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:36.264092 (+ 33us) server_negotiation.cc:407] Connection header received
0809 08:51:36.264573 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:36.264581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:36.264612 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:36.264772 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:36.265436 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:36.265440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:36.265440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:36.265445 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:36.265488 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:36.265502 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:36.265570 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:36.265606 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:51:36.265767 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260809 08:51:36.266808 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:36.267264 11435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113: Bootstrap starting.
I20260809 08:51:36.267874 11435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:36.268231 11435 log.cc:824] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:36.269013 11435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113: No bootstrap required, opened a new log
I20260809 08:51:36.269095 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:36.268019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.213:44723 (local address 127.0.0.1:42459)
0809 08:51:36.268055 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:36.268057 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:36.268284 (+ 227us) server_negotiation.cc:407] Connection header received
0809 08:51:36.268314 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:36.268315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:36.268319 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:36.268377 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:36.268867 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:36.268868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:36.268868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:36.268872 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:36.268883 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:36.268886 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:36.268956 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:36.269018 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 08:51:36.269027 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:36.270762 11435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 08:51:36.270884 11435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:36.270916 11435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70bdffd49c934159a22598b8d3bf6113, State: Initialized, Role: FOLLOWER
I20260809 08:51:36.271029 11435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 08:51:36.271100 11435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:36.271130 11435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:36.271176 11435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:36.271663 11435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 08:51:36.271757 11435 leader_election.cc:302] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 70bdffd49c934159a22598b8d3bf6113; no voters:
I20260809 08:51:36.271905 11435 leader_election.cc:288] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:36.271938 11447 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:36.272081 11447 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [term 1 LEADER]: Becoming Leader. State: Replica: 70bdffd49c934159a22598b8d3bf6113, State: Running, Role: LEADER
I20260809 08:51:36.272122 11435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:36.272169 11447 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } }
I20260809 08:51:36.272469 11447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70bdffd49c934159a22598b8d3bf6113. Latest consensus state: current_term: 1 leader_uuid: "70bdffd49c934159a22598b8d3bf6113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260809 08:51:36.272505 11450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "70bdffd49c934159a22598b8d3bf6113" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "70bdffd49c934159a22598b8d3bf6113" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42459 } } }
I20260809 08:51:36.272610 11447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:36.272660 11450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:36.273159 11459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:36.273571 11459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:36.274948 11459 catalog_manager.cc:1347] Generated new cluster ID: e95c835c931e474d8fbb8da6f1f69bb4
I20260809 08:51:36.274999 11459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:36.280992 11459 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:36.281131 11459 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:36.284906 11459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 70bdffd49c934159a22598b8d3bf6113: Generated new TSK 0
I20260809 08:51:36.285053 11459 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:36.289165 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11328
2026-08-09T08:51:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1355 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:36 dist-test-slave-ncp9 krb5kdc[11475](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:36 dist-test-slave-ncp9 krb5kdc[11475](info): set up 2 sockets
Aug 09 08:51:36 dist-test-slave-ncp9 krb5kdc[11475](info): commencing operation
krb5kdc: starting...
W20260809 08:51:37.406955 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:37.519778 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37453
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:39389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:37453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:37.598523 11495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:37.598718 11495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:37.598739 11495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:37.599891 11495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:37.599938 11495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:37.599951 11495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:37.599961 11495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:37.599972 11495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:37.601446 11495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:39389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:37453
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:37.601675 11495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:37.601867 11495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:37.606640 11495 server_base.cc:1034] running on GCE node
W20260809 08:51:37.606577 11503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:37.606596 11507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:37.606603 11504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:37.607016 11495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:37.607285 11495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:37.608415 11495 hybrid_clock.cc:648] HybridClock initialized: now 1786265497608408 us; error 28 us; skew 500 ppm
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11475](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265497, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:37.610826 11495 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:37.611204 11495 webserver.cc:460] Webserver started at http://127.10.184.126:46563/ using document root <none> and password file <none>
I20260809 08:51:37.611378 11495 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:37.611418 11495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:37.611505 11495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:37.612462 11495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "27134f14518b4d02b1a523d7f77d6837"
format_stamp: "Formatted at 2026-08-09 08:51:37 on dist-test-slave-ncp9"
server_key: "db46272df2f9e02dc5dd64f2bfaba7ce"
server_key_iv: "f5a68980f2e989cec15110d15bc473e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:37.612754 11495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "27134f14518b4d02b1a523d7f77d6837"
format_stamp: "Formatted at 2026-08-09 08:51:37 on dist-test-slave-ncp9"
server_key: "db46272df2f9e02dc5dd64f2bfaba7ce"
server_key_iv: "f5a68980f2e989cec15110d15bc473e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:37.613961 11495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:51:37.614671 11524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:37.614818 11495 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:37.614900 11495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "27134f14518b4d02b1a523d7f77d6837"
format_stamp: "Formatted at 2026-08-09 08:51:37 on dist-test-slave-ncp9"
server_key: "db46272df2f9e02dc5dd64f2bfaba7ce"
server_key_iv: "f5a68980f2e989cec15110d15bc473e6"
server_key_version: "encryptionkey@0"
I20260809 08:51:37.614954 11495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:37.623538 11495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:37.625135 11495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:37.625259 11495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:37.628834 11495 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:37453
I20260809 08:51:37.628877 11602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:37453 every 8 connection(s)
I20260809 08:51:37.629185 11495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:37.629988 11603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:37.632131 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837: Bootstrap starting.
I20260809 08:51:37.632650 11603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:37.632880 11603 log.cc:824] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:37.633543 11603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837: No bootstrap required, opened a new log
I20260809 08:51:37.633930 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11495
I20260809 08:51:37.634055 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:37.634240 10977 external_mini_cluster.cc:1428] Setting key f16c0d07d8d3ca07eff74ed895818de4
I20260809 08:51:37.635581 11603 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } }
I20260809 08:51:37.635748 11603 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:37.635821 11603 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27134f14518b4d02b1a523d7f77d6837, State: Initialized, Role: FOLLOWER
I20260809 08:51:37.635952 11603 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } }
I20260809 08:51:37.636025 11603 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:37.636049 11603 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:37.636092 11603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:37.637128 11603 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } }
I20260809 08:51:37.637245 11603 leader_election.cc:302] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27134f14518b4d02b1a523d7f77d6837; no voters:
I20260809 08:51:37.637400 11603 leader_election.cc:288] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:37.637436 11609 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:37.637570 11609 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [term 1 LEADER]: Becoming Leader. State: Replica: 27134f14518b4d02b1a523d7f77d6837, State: Running, Role: LEADER
I20260809 08:51:37.637614 11603 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:37.637693 11609 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } }
I20260809 08:51:37.638019 11612 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27134f14518b4d02b1a523d7f77d6837. Latest consensus state: current_term: 1 leader_uuid: "27134f14518b4d02b1a523d7f77d6837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } } }
I20260809 08:51:37.638036 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27134f14518b4d02b1a523d7f77d6837" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27134f14518b4d02b1a523d7f77d6837" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37453 } } }
I20260809 08:51:37.638118 11612 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:37.638127 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:37.639389 11618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:37.640296 11618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11475](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265497, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:37.641876 11618 catalog_manager.cc:1347] Generated new cluster ID: fc0a59aa0d814161b41085850d528367
I20260809 08:51:37.641924 11618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:37.645558 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:37.634715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42717 (local address 127.10.184.126:37453)
0809 08:51:37.634913 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:37.634922 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:37.634954 (+ 32us) server_negotiation.cc:407] Connection header received
0809 08:51:37.635543 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:37.635552 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:37.635614 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:37.635766 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:37.636455 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:37.637870 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:37.638694 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:37.639118 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:37.639202 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:37.641810 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:37.641814 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:37.641814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:37.641825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:37.643670 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:37.644060 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:37.644063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:37.644064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:37.644189 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:37.644386 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:37.644386 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:37.644387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:37.644647 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:37.644796 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:37.645269 (+ 473us) server_negotiation.cc:299] Negotiation successful
0809 08:51:37.645362 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":60,"threads_started":1}
I20260809 08:51:37.646344 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:37.647573 11618 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:37.648196 11618 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:37.651101 11618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27134f14518b4d02b1a523d7f77d6837: Generated new TSK 0
I20260809 08:51:37.651258 11618 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:37.654294 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:37.647709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33651 (local address 127.10.184.126:37453)
0809 08:51:37.647740 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:37.647742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:37.647999 (+ 257us) server_negotiation.cc:407] Connection header received
0809 08:51:37.648047 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:37.648047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:37.648206 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:37.648260 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:37.648857 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:37.649634 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:37.650632 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:37.650899 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:37.650942 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:37.652089 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:37.652090 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:37.652091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:37.652097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:37.653295 (+ 1198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:37.653611 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:37.653612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:37.653613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:37.653670 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:37.653855 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:37.653856 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:37.653856 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:37.653953 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:37.654046 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:37.654162 (+ 116us) server_negotiation.cc:299] Negotiation successful
0809 08:51:37.654219 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:37.656023 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11495
W20260809 08:51:37.675792 11472 connection.cc:535] client connection to 127.10.184.126:37453 recv error: Network error: failed to read from TLS socket (remote: 127.10.184.126:37453): Connection reset by peer (error 104)
2026-08-09T08:51:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1373 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11645](info): set up 2 sockets
Aug 09 08:51:37 dist-test-slave-ncp9 krb5kdc[11645](info): commencing operation
krb5kdc: starting...
W20260809 08:51:38.784737 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:38 dist-test-slave-ncp9 krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265498, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:38.893273 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37235
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:45901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:37235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:38.972637 11662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:38.972824 11662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:38.972853 11662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:38.973989 11662 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:38.974030 11662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:38.974049 11662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:38.974063 11662 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:38.974080 11662 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:38.975557 11662 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:45901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:37235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:37235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11662
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:38.975780 11662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:38.975977 11662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:38.980547 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:38.980578 11670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:38.980602 11662 server_base.cc:1034] running on GCE node
W20260809 08:51:38.980553 11672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:38.980988 11662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:38.981256 11662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:38.982394 11662 hybrid_clock.cc:648] HybridClock initialized: now 1786265498982376 us; error 30 us; skew 500 ppm
Aug 09 08:51:38 dist-test-slave-ncp9 krb5kdc[11645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265498, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:38.987426 11662 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:38.987824 11662 webserver.cc:460] Webserver started at http://127.10.184.126:46445/ using document root <none> and password file <none>
I20260809 08:51:38.988027 11662 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:38.988070 11662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:38.988155 11662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:38.989142 11662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4"
format_stamp: "Formatted at 2026-08-09 08:51:38 on dist-test-slave-ncp9"
server_key: "defc59e778cb2ca3ce26ae65b39c0a55"
server_key_iv: "340dab3bf5ddea18769af0e3ac0a33a4"
server_key_version: "encryptionkey@0"
I20260809 08:51:38.989429 11662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4"
format_stamp: "Formatted at 2026-08-09 08:51:38 on dist-test-slave-ncp9"
server_key: "defc59e778cb2ca3ce26ae65b39c0a55"
server_key_iv: "340dab3bf5ddea18769af0e3ac0a33a4"
server_key_version: "encryptionkey@0"
I20260809 08:51:38.990561 11662 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:38.991159 11683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:38.991305 11662 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:38.991384 11662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4"
format_stamp: "Formatted at 2026-08-09 08:51:38 on dist-test-slave-ncp9"
server_key: "defc59e778cb2ca3ce26ae65b39c0a55"
server_key_iv: "340dab3bf5ddea18769af0e3ac0a33a4"
server_key_version: "encryptionkey@0"
I20260809 08:51:38.991444 11662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:38.998656 11662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:39.000619 11662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:39.000749 11662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:39.004374 11662 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:37235
I20260809 08:51:39.004415 11760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:37235 every 8 connection(s)
I20260809 08:51:39.004710 11662 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:39.005558 11762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:39.007253 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11662
I20260809 08:51:39.007349 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:39.007507 10977 external_mini_cluster.cc:1428] Setting key f4d673cd52e10689e40c844f99b6207f
I20260809 08:51:39.007743 11762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4: Bootstrap starting.
I20260809 08:51:39.008355 11762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:39.008605 11762 log.cc:824] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:39.009249 11762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4: No bootstrap required, opened a new log
I20260809 08:51:39.010936 11762 raft_consensus.cc:348] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } }
I20260809 08:51:39.011062 11762 raft_consensus.cc:374] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:39.011094 11762 raft_consensus.cc:729] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebe93fe480ee40fe8bdd744bb1ca33c4, State: Initialized, Role: FOLLOWER
I20260809 08:51:39.011214 11762 consensus_queue.cc:260] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } }
I20260809 08:51:39.011281 11762 raft_consensus.cc:388] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:39.011313 11762 raft_consensus.cc:482] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:39.011354 11762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:39.011838 11762 raft_consensus.cc:504] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } }
I20260809 08:51:39.011933 11762 leader_election.cc:302] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ebe93fe480ee40fe8bdd744bb1ca33c4; no voters:
I20260809 08:51:39.012077 11762 leader_election.cc:288] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:39.012108 11772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:39.012228 11772 raft_consensus.cc:686] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [term 1 LEADER]: Becoming Leader. State: Replica: ebe93fe480ee40fe8bdd744bb1ca33c4, State: Running, Role: LEADER
I20260809 08:51:39.012310 11772 consensus_queue.cc:237] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } }
I20260809 08:51:39.012336 11762 sys_catalog.cc:564] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:39.012650 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:39.007951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38461 (local address 127.10.184.126:37235)
0809 08:51:39.008136 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:39.008144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:39.008181 (+ 37us) server_negotiation.cc:407] Connection header received
0809 08:51:39.008616 (+ 435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:39.008622 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:39.008671 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:39.008807 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:39.009042 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:39.009955 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:39.010721 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:39.011135 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:39.011252 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:39.011845 (+ 593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:39.011849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:39.011849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:39.012026 (+ 177us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:39.012061 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:39.012076 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:39.012144 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:39.012406 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 08:51:39.012434 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260809 08:51:39.012710 11775 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ebe93fe480ee40fe8bdd744bb1ca33c4. Latest consensus state: current_term: 1 leader_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } } }
I20260809 08:51:39.012780 11775 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:39.012935 11774 sys_catalog.cc:455] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ebe93fe480ee40fe8bdd744bb1ca33c4" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 37235 } } }
I20260809 08:51:39.012988 11774 sys_catalog.cc:458] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:39.013525 11778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:39.013608 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:39.014002 11778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:39.015729 11778 catalog_manager.cc:1347] Generated new cluster ID: bbbc39fafeb949048cbb9e4741285cfd
I20260809 08:51:39.015784 11778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:39.017966 11770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:39.015125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51813 (local address 127.10.184.126:37235)
0809 08:51:39.015152 (+ 27us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:39.015154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:39.015334 (+ 180us) server_negotiation.cc:407] Connection header received
0809 08:51:39.015387 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:39.015388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:39.015396 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:39.015449 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:39.015587 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:39.016124 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:39.016821 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:39.017127 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:39.017189 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:39.017750 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:39.017751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:39.017751 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:39.017755 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:39.017768 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:39.017772 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:39.017816 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:39.017891 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 08:51:39.017899 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12}
I20260809 08:51:39.024425 11778 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:39.025098 11778 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:39.039858 11778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ebe93fe480ee40fe8bdd744bb1ca33c4: Generated new TSK 0
I20260809 08:51:39.040015 11778 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:39.046536 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11662
W20260809 08:51:39.057657 11643 connection.cc:535] client connection to 127.10.184.126:37235 recv error: Network error: failed to read from TLS socket (remote: 127.10.184.126:37235): Connection reset by peer (error 104)
2026-08-09T08:51:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:39 dist-test-slave-ncp9 krb5kdc[11808](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:39 dist-test-slave-ncp9 krb5kdc[11808](info): set up 2 sockets
Aug 09 08:51:39 dist-test-slave-ncp9 krb5kdc[11808](info): commencing operation
krb5kdc: starting...
W20260809 08:51:40.166433 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:40 dist-test-slave-ncp9 krb5kdc[11808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265500, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:40.275297 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38551
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:46845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:38551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:40.353634 11822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:40.353829 11822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:40.353859 11822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:40.354966 11822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:40.355005 11822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:40.355024 11822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:40.355041 11822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:40.355055 11822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:40.356467 11822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:46845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:38551
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:38551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:40.356704 11822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:40.356912 11822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:40.361423 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:40.361487 11833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:40.361510 11836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:40.361590 11822 server_base.cc:1034] running on GCE node
I20260809 08:51:40.361852 11822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:40.362114 11822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:40.363265 11822 hybrid_clock.cc:648] HybridClock initialized: now 1786265500363263 us; error 34 us; skew 500 ppm
Aug 09 08:51:40 dist-test-slave-ncp9 krb5kdc[11808](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265500, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:40.365532 11822 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:40.365914 11822 webserver.cc:460] Webserver started at http://127.10.184.126:34359/ using document root <none> and password file <none>
I20260809 08:51:40.366101 11822 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:40.366154 11822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:40.366276 11822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:40.367246 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "ec5b8c3c0d814793a08891c763c93450"
format_stamp: "Formatted at 2026-08-09 08:51:40 on dist-test-slave-ncp9"
server_key: "629ded6a28e50fdf5ee9b707e4ae59b0"
server_key_iv: "2435992d8b966759b36f58b957111a76"
server_key_version: "encryptionkey@0"
I20260809 08:51:40.367569 11822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "ec5b8c3c0d814793a08891c763c93450"
format_stamp: "Formatted at 2026-08-09 08:51:40 on dist-test-slave-ncp9"
server_key: "629ded6a28e50fdf5ee9b707e4ae59b0"
server_key_iv: "2435992d8b966759b36f58b957111a76"
server_key_version: "encryptionkey@0"
I20260809 08:51:40.368781 11822 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:51:40.369521 11850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:40.369666 11822 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:40.369748 11822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "ec5b8c3c0d814793a08891c763c93450"
format_stamp: "Formatted at 2026-08-09 08:51:40 on dist-test-slave-ncp9"
server_key: "629ded6a28e50fdf5ee9b707e4ae59b0"
server_key_iv: "2435992d8b966759b36f58b957111a76"
server_key_version: "encryptionkey@0"
I20260809 08:51:40.369812 11822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:40.372845 11822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:40.374511 11822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:40.374626 11822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:40.378170 11822 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:38551
I20260809 08:51:40.378204 11936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:38551 every 8 connection(s)
I20260809 08:51:40.378481 11822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:40.378895 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 11822
I20260809 08:51:40.378997 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:40.379151 10977 external_mini_cluster.cc:1428] Setting key 48b7c74002cf25f574c39d2dce84739a
I20260809 08:51:40.379490 11937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:40.381314 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:40.379606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43387 (local address 127.10.184.126:38551)
0809 08:51:40.379775 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:40.379782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:40.379806 (+ 24us) server_negotiation.cc:407] Connection header received
0809 08:51:40.380200 (+ 394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:40.380206 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:40.380230 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:40.380336 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:40.380887 (+ 551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:40.380915 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:40.380916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:40.380921 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:40.380962 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:40.380972 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:40.381024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:40.381104 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 08:51:40.381135 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260809 08:51:40.382264 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:40.382611 11937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450: Bootstrap starting.
I20260809 08:51:40.383191 11937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:40.383520 11937 log.cc:824] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:40.384295 11937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450: No bootstrap required, opened a new log
I20260809 08:51:40.384495 11942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:40.383595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55981 (local address 127.10.184.126:38551)
0809 08:51:40.383630 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:40.383631 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:40.383643 (+ 12us) server_negotiation.cc:407] Connection header received
0809 08:51:40.383732 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:40.383733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:40.383738 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:40.383780 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:40.384311 (+ 531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:40.384312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:40.384312 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:40.384316 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:40.384328 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:40.384332 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:40.384379 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:40.384425 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 08:51:40.384437 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:51:40.386224 11937 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } }
I20260809 08:51:40.386351 11937 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:40.386373 11937 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec5b8c3c0d814793a08891c763c93450, State: Initialized, Role: FOLLOWER
I20260809 08:51:40.386462 11937 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } }
I20260809 08:51:40.386554 11937 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:40.386579 11937 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:40.386605 11937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:40.387059 11937 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } }
I20260809 08:51:40.387156 11937 leader_election.cc:302] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec5b8c3c0d814793a08891c763c93450; no voters:
I20260809 08:51:40.387351 11937 leader_election.cc:288] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:40.387382 11952 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:40.387485 11952 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [term 1 LEADER]: Becoming Leader. State: Replica: ec5b8c3c0d814793a08891c763c93450, State: Running, Role: LEADER
I20260809 08:51:40.387598 11937 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:40.387629 11952 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } }
I20260809 08:51:40.387988 11953 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec5b8c3c0d814793a08891c763c93450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } } }
I20260809 08:51:40.388012 11954 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec5b8c3c0d814793a08891c763c93450. Latest consensus state: current_term: 1 leader_uuid: "ec5b8c3c0d814793a08891c763c93450" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec5b8c3c0d814793a08891c763c93450" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 38551 } } }
I20260809 08:51:40.388155 11953 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:40.388166 11954 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:40.388629 11964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:40.389137 11964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:40.390472 11964 catalog_manager.cc:1347] Generated new cluster ID: 3ce6dd01f8f740f189795383239babaf
I20260809 08:51:40.390524 11964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:40.434840 11964 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:40.435005 11964 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:40.438627 11964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec5b8c3c0d814793a08891c763c93450: Generated new TSK 0
I20260809 08:51:40.438791 11964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:40.440881 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 11822
2026-08-09T08:51:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:40 dist-test-slave-ncp9 krb5kdc[11983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:40 dist-test-slave-ncp9 krb5kdc[11983](info): set up 2 sockets
Aug 09 08:51:40 dist-test-slave-ncp9 krb5kdc[11983](info): commencing operation
krb5kdc: starting...
W20260809 08:51:41.560930 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[11983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:41.671231 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40675
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:41.750900 12006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:41.751080 12006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:41.751101 12006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:41.752202 12006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:41.752240 12006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:41.752252 12006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:41.752275 12006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:41.752297 12006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:41.753775 12006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:41.753985 12006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:41.754180 12006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:41.758831 12014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:41.758862 12016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:41.758865 12019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:41.759022 12006 server_base.cc:1034] running on GCE node
I20260809 08:51:41.759262 12006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:41.759532 12006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:41.760668 12006 hybrid_clock.cc:648] HybridClock initialized: now 1786265501760652 us; error 31 us; skew 500 ppm
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[11983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265501, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:41.763094 12006 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:41.763492 12006 webserver.cc:460] Webserver started at http://127.10.184.126:46789/ using document root <none> and password file <none>
I20260809 08:51:41.763705 12006 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:41.763753 12006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:41.763877 12006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:41.764881 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "1f45ee4f416140aaa1460c937678548a"
format_stamp: "Formatted at 2026-08-09 08:51:41 on dist-test-slave-ncp9"
server_key: "991432f987764355c611e2a3851e1464"
server_key_iv: "583ecf42967ba8fc1befd2f346fe2d61"
server_key_version: "encryptionkey@0"
I20260809 08:51:41.765275 12006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "1f45ee4f416140aaa1460c937678548a"
format_stamp: "Formatted at 2026-08-09 08:51:41 on dist-test-slave-ncp9"
server_key: "991432f987764355c611e2a3851e1464"
server_key_iv: "583ecf42967ba8fc1befd2f346fe2d61"
server_key_version: "encryptionkey@0"
I20260809 08:51:41.766479 12006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 08:51:41.767118 12028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:41.767275 12006 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 08:51:41.767373 12006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "1f45ee4f416140aaa1460c937678548a"
format_stamp: "Formatted at 2026-08-09 08:51:41 on dist-test-slave-ncp9"
server_key: "991432f987764355c611e2a3851e1464"
server_key_iv: "583ecf42967ba8fc1befd2f346fe2d61"
server_key_version: "encryptionkey@0"
I20260809 08:51:41.767437 12006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:41.779098 12006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:41.781152 12006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:41.781307 12006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:41.786597 12006 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:40675
I20260809 08:51:41.786626 12099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:40675 every 8 connection(s)
I20260809 08:51:41.786913 12006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:41.787724 12101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:41.789870 12101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a: Bootstrap starting.
I20260809 08:51:41.790403 12101 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:41.790654 12101 log.cc:824] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:41.791273 12101 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a: No bootstrap required, opened a new log
I20260809 08:51:41.792783 12101 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } }
I20260809 08:51:41.792943 12101 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:41.792980 12101 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f45ee4f416140aaa1460c937678548a, State: Initialized, Role: FOLLOWER
I20260809 08:51:41.793114 12101 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } }
I20260809 08:51:41.793193 12101 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:41.793236 12101 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:41.793282 12101 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:41.794060 12101 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } }
I20260809 08:51:41.794155 12101 leader_election.cc:302] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f45ee4f416140aaa1460c937678548a; no voters:
I20260809 08:51:41.794312 12101 leader_election.cc:288] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:41.794348 12105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:41.794494 12105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [term 1 LEADER]: Becoming Leader. State: Replica: 1f45ee4f416140aaa1460c937678548a, State: Running, Role: LEADER
I20260809 08:51:41.794544 12101 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:41.794582 12105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } }
I20260809 08:51:41.794950 12106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f45ee4f416140aaa1460c937678548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } } }
I20260809 08:51:41.794963 12107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f45ee4f416140aaa1460c937678548a. Latest consensus state: current_term: 1 leader_uuid: "1f45ee4f416140aaa1460c937678548a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f45ee4f416140aaa1460c937678548a" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 40675 } } }
I20260809 08:51:41.795058 12107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:41.795053 12106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:41.795311 12115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:41.795379 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12006
I20260809 08:51:41.795501 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:41.795694 10977 external_mini_cluster.cc:1428] Setting key b33e18d3ad5c697fec3bc889af343e4e
I20260809 08:51:41.796010 12115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:41.797573 12115 catalog_manager.cc:1347] Generated new cluster ID: 4b471752b9c447489663fd60006a8ebe
I20260809 08:51:41.797626 12115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[11983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:41.805685 12115 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:41.806392 12115 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:41.806875 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:41.796117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57283 (local address 127.10.184.126:40675)
0809 08:51:41.796307 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:41.796316 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:41.796347 (+ 31us) server_negotiation.cc:407] Connection header received
0809 08:51:41.796626 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:41.796632 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:41.796699 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:41.796834 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:41.797601 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:41.798953 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:41.800056 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:41.800652 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:41.800741 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:41.803386 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:41.803390 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:41.803390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:41.803402 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:41.805324 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:41.805739 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:41.805742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:41.805742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:41.805844 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:41.806018 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:41.806019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:41.806019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:41.806269 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:41.806417 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:41.806510 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:51:41.806651 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":149,"thread_start_us":80,"threads_started":1}
I20260809 08:51:41.807705 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:41.810901 12115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f45ee4f416140aaa1460c937678548a: Generated new TSK 0
I20260809 08:51:41.811059 12115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:41.814419 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:41.808964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58035 (local address 127.10.184.126:40675)
0809 08:51:41.808998 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:41.809000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:41.809113 (+ 113us) server_negotiation.cc:407] Connection header received
0809 08:51:41.809169 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:41.809170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:41.809179 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:41.809222 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:41.809683 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:41.810161 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:41.810924 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:41.811219 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:41.811275 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:41.812277 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:41.812278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:41.812278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:41.812285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:41.813477 (+ 1192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:41.813769 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:41.813770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:41.813770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:41.813821 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:41.813985 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:41.813986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:41.813987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:41.814085 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:41.814170 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:41.814271 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 08:51:41.814342 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:41.816231 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12006
2026-08-09T08:51:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[12141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[12141](info): set up 2 sockets
Aug 09 08:51:41 dist-test-slave-ncp9 krb5kdc[12141](info): commencing operation
krb5kdc: starting...
W20260809 08:51:42.938884 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:43 dist-test-slave-ncp9 krb5kdc[12141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265503, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:43.048079 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:46685
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:38799
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:46685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:43.127350 12161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:43.127550 12161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:43.127579 12161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:43.128705 12161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:43.128747 12161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:43.128767 12161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:43.128782 12161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:43.128798 12161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:43.130306 12161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:38799
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:46685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:46685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:43.130544 12161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:43.130741 12161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:43.135427 12167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:43.135434 12168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:43.135434 12173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:43.135461 12161 server_base.cc:1034] running on GCE node
I20260809 08:51:43.135936 12161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:43.136215 12161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:43.137343 12161 hybrid_clock.cc:648] HybridClock initialized: now 1786265503137326 us; error 33 us; skew 500 ppm
Aug 09 08:51:43 dist-test-slave-ncp9 krb5kdc[12141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265503, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:43.139653 12161 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:43.140069 12161 webserver.cc:460] Webserver started at http://127.10.184.126:32845/ using document root <none> and password file <none>
I20260809 08:51:43.140259 12161 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:43.140306 12161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:43.140429 12161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:43.141455 12161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "971621460a0b4dec9e5348149a5139b0"
format_stamp: "Formatted at 2026-08-09 08:51:43 on dist-test-slave-ncp9"
server_key: "f96ce2226841e192ea3d5cdecf341964"
server_key_iv: "099fb034b31dd6e8bcbe41e31b1a2a1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:43.141782 12161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "971621460a0b4dec9e5348149a5139b0"
format_stamp: "Formatted at 2026-08-09 08:51:43 on dist-test-slave-ncp9"
server_key: "f96ce2226841e192ea3d5cdecf341964"
server_key_iv: "099fb034b31dd6e8bcbe41e31b1a2a1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:43.143013 12161 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:43.143699 12183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:43.143857 12161 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:43.143945 12161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "971621460a0b4dec9e5348149a5139b0"
format_stamp: "Formatted at 2026-08-09 08:51:43 on dist-test-slave-ncp9"
server_key: "f96ce2226841e192ea3d5cdecf341964"
server_key_iv: "099fb034b31dd6e8bcbe41e31b1a2a1d"
server_key_version: "encryptionkey@0"
I20260809 08:51:43.144004 12161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:43.155231 12161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:43.157224 12161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:43.157351 12161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:43.160878 12161 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:46685
I20260809 08:51:43.160945 12255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:46685 every 8 connection(s)
I20260809 08:51:43.161252 12161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:43.162120 12256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:43.162158 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12161
I20260809 08:51:43.162266 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:43.162469 10977 external_mini_cluster.cc:1428] Setting key d346c808426bcbb8c01776f4e51e334e
I20260809 08:51:43.165087 12256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0: Bootstrap starting.
I20260809 08:51:43.165768 12256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:43.166040 12256 log.cc:824] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:43.166720 12256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0: No bootstrap required, opened a new log
I20260809 08:51:43.167641 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:43.162974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58551 (local address 127.10.184.126:46685)
0809 08:51:43.163143 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:43.163150 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:43.163177 (+ 27us) server_negotiation.cc:407] Connection header received
0809 08:51:43.163602 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:43.163609 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:43.163651 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:43.163765 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:43.163990 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:43.164837 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:43.165876 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:43.166290 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:43.166369 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:43.167011 (+ 642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:43.167015 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:43.167015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:43.167203 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:43.167241 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:43.167255 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:43.167319 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:43.167370 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 08:51:43.167430 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":62,"threads_started":1}
I20260809 08:51:43.168488 12256 raft_consensus.cc:348] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } }
I20260809 08:51:43.168632 12256 raft_consensus.cc:374] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:43.168664 12256 raft_consensus.cc:729] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 971621460a0b4dec9e5348149a5139b0, State: Initialized, Role: FOLLOWER
I20260809 08:51:43.168778 12256 consensus_queue.cc:260] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } }
I20260809 08:51:43.168860 12256 raft_consensus.cc:388] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:43.168923 12256 raft_consensus.cc:482] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:43.168920 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:43.168965 12256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:43.169440 12256 raft_consensus.cc:504] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } }
I20260809 08:51:43.169516 12256 leader_election.cc:302] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 971621460a0b4dec9e5348149a5139b0; no voters:
I20260809 08:51:43.169636 12256 leader_election.cc:288] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:43.169704 12262 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:43.169837 12262 raft_consensus.cc:686] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [term 1 LEADER]: Becoming Leader. State: Replica: 971621460a0b4dec9e5348149a5139b0, State: Running, Role: LEADER
I20260809 08:51:43.169844 12256 sys_catalog.cc:564] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:43.169929 12262 consensus_queue.cc:237] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } }
I20260809 08:51:43.170238 12265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 971621460a0b4dec9e5348149a5139b0. Latest consensus state: current_term: 1 leader_uuid: "971621460a0b4dec9e5348149a5139b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } } }
I20260809 08:51:43.170307 12265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:43.170317 12270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "971621460a0b4dec9e5348149a5139b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "971621460a0b4dec9e5348149a5139b0" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 46685 } } }
I20260809 08:51:43.170403 12270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:43.170528 12280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:43.170994 12280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:43.172312 12292 mvcc.cc:205] Tried to move back new op lower bound from 7316543500992200704 to 7316543500984356864. Current Snapshot: MvccSnapshot[applied={T|T < 7316543500992200704}]
I20260809 08:51:43.172763 12280 catalog_manager.cc:1347] Generated new cluster ID: 35932eb588f84e6ca11f1048eb13c908
I20260809 08:51:43.172809 12280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:43.173615 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:43.170251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33153 (local address 127.10.184.126:46685)
0809 08:51:43.170281 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:43.170282 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:43.170555 (+ 273us) server_negotiation.cc:407] Connection header received
0809 08:51:43.170651 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:43.170651 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:43.170673 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:43.170701 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:43.170889 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:43.171511 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:43.172404 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:43.172690 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:43.172758 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:43.173398 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:43.173399 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:43.173399 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:43.173403 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:43.173417 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:43.173421 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:43.173469 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:43.173529 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 08:51:43.173549 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260809 08:51:43.184496 12280 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:43.185002 12280 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:43.192555 12280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 971621460a0b4dec9e5348149a5139b0: Generated new TSK 0
I20260809 08:51:43.192709 12280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:43.199342 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12161
2026-08-09T08:51:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1385 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:43 dist-test-slave-ncp9 krb5kdc[12298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:43 dist-test-slave-ncp9 krb5kdc[12298](info): set up 2 sockets
Aug 09 08:51:43 dist-test-slave-ncp9 krb5kdc[12298](info): commencing operation
krb5kdc: starting...
W20260809 08:51:44.331410 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:44 dist-test-slave-ncp9 krb5kdc[12298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265504, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:44.445050 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44761
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:43281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:44.539590 12317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:44.539798 12317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:44.539829 12317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:44.540987 12317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:44.541034 12317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:44.541054 12317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:44.541070 12317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:44.541086 12317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:44.542559 12317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:43281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44761
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:44.542805 12317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:44.542994 12317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:44.548085 12329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:44.548092 12324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:44.548360 12317 server_base.cc:1034] running on GCE node
W20260809 08:51:44.548105 12323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:44.548650 12317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:44.548942 12317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:44.550081 12317 hybrid_clock.cc:648] HybridClock initialized: now 1786265504550070 us; error 27 us; skew 500 ppm
Aug 09 08:51:44 dist-test-slave-ncp9 krb5kdc[12298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265504, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:44.552379 12317 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:44.552795 12317 webserver.cc:460] Webserver started at http://127.10.184.126:39815/ using document root <none> and password file <none>
I20260809 08:51:44.553026 12317 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:44.553076 12317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:44.553207 12317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:44.554167 12317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "3f1358aad80340ea9e706ea164615059"
format_stamp: "Formatted at 2026-08-09 08:51:44 on dist-test-slave-ncp9"
server_key: "b7be5596ebed61620fcd02bba563b9f2"
server_key_iv: "3e3e122b62d8b2a045fa4343188093c9"
server_key_version: "encryptionkey@0"
I20260809 08:51:44.554500 12317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "3f1358aad80340ea9e706ea164615059"
format_stamp: "Formatted at 2026-08-09 08:51:44 on dist-test-slave-ncp9"
server_key: "b7be5596ebed61620fcd02bba563b9f2"
server_key_iv: "3e3e122b62d8b2a045fa4343188093c9"
server_key_version: "encryptionkey@0"
I20260809 08:51:44.555691 12317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:51:44.556316 12344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:44.556457 12317 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:44.556522 12317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "3f1358aad80340ea9e706ea164615059"
format_stamp: "Formatted at 2026-08-09 08:51:44 on dist-test-slave-ncp9"
server_key: "b7be5596ebed61620fcd02bba563b9f2"
server_key_iv: "3e3e122b62d8b2a045fa4343188093c9"
server_key_version: "encryptionkey@0"
I20260809 08:51:44.556588 12317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:44.559682 12317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:44.561369 12317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:44.561483 12317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:44.565095 12317 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44761
I20260809 08:51:44.565137 12414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44761 every 8 connection(s)
I20260809 08:51:44.565423 12317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:44.566319 12415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:44.568346 12415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: Bootstrap starting.
I20260809 08:51:44.568886 12415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:44.569181 12415 log.cc:824] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:44.569795 12415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: No bootstrap required, opened a new log
I20260809 08:51:44.570261 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12317
I20260809 08:51:44.570369 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:44.570525 10977 external_mini_cluster.cc:1428] Setting key 9d947fbcc1c74b4825e728918f4993d8
I20260809 08:51:44.572120 12415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } }
I20260809 08:51:44.572283 12415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 08:51:44.572356 12415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f1358aad80340ea9e706ea164615059, State: Initialized, Role: FOLLOWER
I20260809 08:51:44.572731 12415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } }
I20260809 08:51:44.572834 12415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 08:51:44.572868 12415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 08:51:44.572945 12415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:44.573262 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:44.570981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41633 (local address 127.10.184.126:44761)
0809 08:51:44.571167 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:44.571177 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:44.571212 (+ 35us) server_negotiation.cc:407] Connection header received
0809 08:51:44.572052 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:44.572062 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:44.572097 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:44.572247 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:44.572791 (+ 544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:44.572795 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:44.572795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:44.572800 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:44.572835 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:44.572845 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:44.572914 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:44.573032 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 08:51:44.573076 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260809 08:51:44.573549 12415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } }
I20260809 08:51:44.573637 12415 leader_election.cc:302] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f1358aad80340ea9e706ea164615059; no voters:
I20260809 08:51:44.573836 12415 leader_election.cc:288] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 08:51:44.573884 12420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:44.574051 12420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [term 1 LEADER]: Becoming Leader. State: Replica: 3f1358aad80340ea9e706ea164615059, State: Running, Role: LEADER
I20260809 08:51:44.574100 12415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:44.574162 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:44.574182 12420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } }
I20260809 08:51:44.574476 12420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f1358aad80340ea9e706ea164615059. Latest consensus state: current_term: 1 leader_uuid: "3f1358aad80340ea9e706ea164615059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } } }
I20260809 08:51:44.574551 12420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:44.574719 12422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f1358aad80340ea9e706ea164615059" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f1358aad80340ea9e706ea164615059" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44761 } } }
I20260809 08:51:44.574791 12422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059 [sys.catalog]: This master's current role is: LEADER
W20260809 08:51:44.575939 12449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:44.575999 12449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:44.576241 12427 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:44.576412 12427 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:44.576537 12419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:44.575733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46415 (local address 127.10.184.126:44761)
0809 08:51:44.575767 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:44.575769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:44.575779 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:44.575825 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:44.575826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:44.575831 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:44.575857 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:44.576347 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:44.576349 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:44.576349 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 08:51:44.576353 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:44.576368 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 08:51:44.576372 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:44.576437 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:44.576478 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:51:44.576490 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 08:51:44.577833 12427 catalog_manager.cc:1347] Generated new cluster ID: 663cc29bd2d5414b8d2c097ddfc75d6e
I20260809 08:51:44.577886 12427 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:44.595928 12427 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:44.596079 12427 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:44.602349 12427 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f1358aad80340ea9e706ea164615059: Generated new TSK 0
I20260809 08:51:44.602502 12427 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:44.606590 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12317
2026-08-09T08:51:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1399 ms)
[----------] 18 tests from AuthTokenIssuingTest (25055 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:44 dist-test-slave-ncp9 krb5kdc[12455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:44 dist-test-slave-ncp9 krb5kdc[12455](info): set up 2 sockets
Aug 09 08:51:44 dist-test-slave-ncp9 krb5kdc[12455](info): commencing operation
krb5kdc: starting...
W20260809 08:51:45.742699 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:45 dist-test-slave-ncp9 krb5kdc[12455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:45.875669 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39203
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:45.989598 12470 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:45.989877 12470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:45.989905 12470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:45.989946 12470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:45.991370 12470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:45.991423 12470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:45.991446 12470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:45.991475 12470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:45.991514 12470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:45.993528 12470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:39203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:45.993827 12470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:45.994112 12470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:46.000558 12482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.000533 12479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.000948 12484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:46.000953 12470 server_base.cc:1034] running on GCE node
I20260809 08:51:46.001331 12470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:46.001737 12470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:46.002911 12470 hybrid_clock.cc:648] HybridClock initialized: now 1786265506002887 us; error 35 us; skew 500 ppm
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:46.006222 12470 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:46.006929 12470 webserver.cc:460] Webserver started at http://127.10.184.126:34277/ using document root <none> and password file <none>
I20260809 08:51:46.007248 12470 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:46.007365 12470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:46.007550 12470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:46.009068 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "564d7e5bdbbe4c2ea642596e964c3865"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "602c729ad10f417645823fd268b0cbe2"
server_key_iv: "0438720f5d796611e2d1a52d3f168887"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.009582 12470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "564d7e5bdbbe4c2ea642596e964c3865"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "602c729ad10f417645823fd268b0cbe2"
server_key_iv: "0438720f5d796611e2d1a52d3f168887"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.011898 12470 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 08:51:46.013090 12494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:46.013360 12470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:46.013512 12470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "564d7e5bdbbe4c2ea642596e964c3865"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "602c729ad10f417645823fd268b0cbe2"
server_key_iv: "0438720f5d796611e2d1a52d3f168887"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.013602 12470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:46.036278 12470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:46.039680 12470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:46.039906 12470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:46.046804 12470 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:39203
I20260809 08:51:46.046880 12584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:39203 every 8 connection(s)
I20260809 08:51:46.047402 12470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:46.048784 12585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:46.051151 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12470
I20260809 08:51:46.051321 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:46.051671 10977 external_mini_cluster.cc:1428] Setting key 4a0658b0fb256b5c6fa815f8429ae1c8
I20260809 08:51:46.051745 12585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.125@KRBTEST.COM; defaulting to no policy
I20260809 08:51:46.062248 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.053182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39203 (local address 127.0.0.1:43880)
0809 08:51:46.053494 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.053523 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.053966 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.054228 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.054231 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.054482 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.054627 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.054632 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.055911 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.055912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.056661 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.056664 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.057390 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.057391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.057460 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.057911 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.057920 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.059199 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.061006 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.061012 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.061014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.061206 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.061390 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.061391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.061391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.061470 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.061786 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.061791 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.061848 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.061922 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.062051 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"spinlock_wait_cycles":14592,"thread_start_us":139,"threads_started":1}
I20260809 08:51:46.062252 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.053187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43880 (local address 127.10.184.126:39203)
0809 08:51:46.053446 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.053456 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.053603 (+ 147us) server_negotiation.cc:407] Connection header received
0809 08:51:46.054115 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.054117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.054133 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.054211 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.054723 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.055809 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.056753 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.057279 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.057406 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.059284 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.059293 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.059293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.059301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.060925 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.061265 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.061267 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.061267 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.061344 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.061518 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.061519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.061519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.061727 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.061789 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.062005 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.062115 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":193,"thread_start_us":137,"threads_started":1}
I20260809 08:51:46.062894 12585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:46.063206 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.063103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49854)
0809 08:51:46.063143 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.063188 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:51:46.063374 12495 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.125@KRBTEST.COM" created.
I20260809 08:51:46.063820 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.063721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49868)
0809 08:51:46.063766 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.063798 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 08:51:46.063916 12585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:42029: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
I20260809 08:51:46.077044 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:42029
--webserver_interface=127.10.184.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:51:46.113121 12585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } attempt: 1
I20260809 08:51:46.113588 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.113465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49872)
0809 08:51:46.113522 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.113557 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 08:51:46.113910 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.113831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49874)
0809 08:51:46.113867 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.113891 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260809 08:51:46.114001 12585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:42029: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
W20260809 08:51:46.156754 12595 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:46.156966 12595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:46.156991 12595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:46.157015 12595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:46.158128 12595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:46.158172 12595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:46.158192 12595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:46.158207 12595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:46.158222 12595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:46.159652 12595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:42029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.184.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:46.159879 12595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:46.160066 12595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:46.164769 12595 server_base.cc:1034] running on GCE node
W20260809 08:51:46.164739 12608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.164738 12605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.164719 12606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:46.165261 12595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:46.165516 12595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:46.166657 12595 hybrid_clock.cc:648] HybridClock initialized: now 1786265506166645 us; error 27 us; skew 500 ppm
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:46.169044 12595 init.cc:377] Logged in from keytab as kudu/127.10.184.125@KRBTEST.COM (short username kudu)
I20260809 08:51:46.169431 12595 webserver.cc:460] Webserver started at http://127.10.184.125:33191/ using document root <none> and password file <none>
I20260809 08:51:46.169618 12595 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:46.169664 12595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:46.169787 12595 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:46.170764 12595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data/instance:
uuid: "f18a90011fac4c169669067ed1baff71"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "dbe97f617a3c3b4c5d7806624391d806"
server_key_iv: "4c9cfb7fbdfa1cae4f7d8a38e07e0785"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.171119 12595 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal/instance:
uuid: "f18a90011fac4c169669067ed1baff71"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "dbe97f617a3c3b4c5d7806624391d806"
server_key_iv: "4c9cfb7fbdfa1cae4f7d8a38e07e0785"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.172320 12595 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:46.172955 12615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:46.173087 12595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:46.173178 12595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "f18a90011fac4c169669067ed1baff71"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "dbe97f617a3c3b4c5d7806624391d806"
server_key_iv: "4c9cfb7fbdfa1cae4f7d8a38e07e0785"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.173242 12595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:46.182142 12585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } attempt: 2
I20260809 08:51:46.182504 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.182382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49878)
0809 08:51:46.182438 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.182475 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:51:46.182775 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.182709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49894)
0809 08:51:46.182740 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.182761 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:51:46.182861 12585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:42029: Network error: Client connection negotiation failed: client connection to 127.10.184.125:42029: connect: Connection refused (error 111)
I20260809 08:51:46.207688 12595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:46.209419 12595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:46.209525 12595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:46.213272 12595 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.125:42029
I20260809 08:51:46.213310 12703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.125:42029 every 8 connection(s)
I20260809 08:51:46.213616 12595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
I20260809 08:51:46.214466 12704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:46.216091 12704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:46.221560 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12595
I20260809 08:51:46.221666 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-1/wal/instance
I20260809 08:51:46.221850 10977 external_mini_cluster.cc:1428] Setting key f1c3554b5016116677522c4869bbf22c
I20260809 08:51:46.224434 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.216749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43894 (local address 127.10.184.126:39203)
0809 08:51:46.216803 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.216805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.217029 (+ 224us) server_negotiation.cc:407] Connection header received
0809 08:51:46.217471 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.217472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.217481 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.217535 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.218064 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.218585 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.219558 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.219766 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.219815 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.221751 (+ 1936us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.221754 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.221754 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.221763 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.223109 (+ 1346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.223507 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.223508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.223509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.223590 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.223863 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.223864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.223865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.223969 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.224022 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.224308 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.224372 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 08:51:46.224677 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.216731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39203 (local address 127.0.0.1:43894)
0809 08:51:46.216911 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.216953 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.217397 (+ 444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.217589 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.217592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.217815 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.218000 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.218004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.218656 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.218656 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.219461 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.219464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.219825 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.219825 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.219919 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.220318 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.220326 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.221628 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.223196 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.223203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.223205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.223419 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.223655 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.223656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.223656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.223803 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.224028 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.224032 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.224187 (+ 155us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.224295 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.224452 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":69,"threads_started":1}
I20260809 08:51:46.225251 12704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.10.184.124@KRBTEST.COM; defaulting to no policy
I20260809 08:51:46.231814 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.225468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49904)
0809 08:51:46.225541 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.225549 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.225579 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.225809 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.225810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.225933 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.225977 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.225978 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.226860 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.226861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.227334 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.227336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.227762 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.227763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.227825 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.228208 (+ 383us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.228212 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.229356 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.230907 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.230908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.230908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.231057 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.231226 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.231226 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.231226 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.231278 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.231518 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.231518 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.231555 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.231616 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.231711 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 08:51:46.231887 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.225547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49904 (local address 127.10.184.125:42029)
0809 08:51:46.225695 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.225699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.225714 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:51:46.225729 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.225731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.225740 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.225798 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.226074 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.226787 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.227399 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.227699 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.227759 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.229429 (+ 1670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.229436 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.229437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.229443 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.230833 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.231123 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.231127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.231127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.231173 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.231326 (+ 153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.231327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.231327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.231452 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.231521 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.231651 (+ 130us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.231762 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.124@KRBTEST.COM" created.
I20260809 08:51:46.232479 12704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:46.232810 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.232690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58142)
0809 08:51:46.232731 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.232785 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:51:46.232972 12619 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
I20260809 08:51:46.233371 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.233282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58150)
0809 08:51:46.233319 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.233348 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260809 08:51:46.233479 12704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40833: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
I20260809 08:51:46.245347 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:40833
--webserver_interface=127.10.184.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:40707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:51:46.274015 12585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } attempt: 3
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:46.280834 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.274339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49908)
0809 08:51:46.274393 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.274404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.274460 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.274583 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.274584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.274734 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.274797 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.274799 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.275504 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.275505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.276063 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.276066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.276461 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.276463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.276561 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.277055 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.277061 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.278362 (+ 1301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.279978 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.279979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.279979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.280153 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.280336 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.280337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.280337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.280381 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.280616 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.280616 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.280654 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.280697 (+ 43us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.280767 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 08:51:46.280936 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.274411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.10.184.125:42029)
0809 08:51:46.274473 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.274475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.274485 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:46.274512 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.274513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.274525 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.274571 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.274874 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.275409 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.276144 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.276367 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.276437 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.278475 (+ 2038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.278477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.278477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.278485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.279895 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.280215 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.280216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.280217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.280277 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.280427 (+ 150us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.280428 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.280428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.280557 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.280633 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.280726 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.280809 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 08:51:46.281435 12585 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:46.281786 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.281693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58162)
0809 08:51:46.281733 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.281762 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:51:46.282130 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.282056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58168)
0809 08:51:46.282093 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.282114 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 08:51:46.282222 12585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40833: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
I20260809 08:51:46.282672 12704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } attempt: 1
I20260809 08:51:46.283023 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.282899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58184)
0809 08:51:46.282938 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.282994 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 08:51:46.283361 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.283224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58196)
0809 08:51:46.283284 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.283314 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 08:51:46.283463 12704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40833: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
I20260809 08:51:46.313374 12585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } attempt: 1
I20260809 08:51:46.313777 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.313643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58204)
0809 08:51:46.313704 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.313744 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 08:51:46.314085 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.314005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58214)
0809 08:51:46.314040 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.314066 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260809 08:51:46.314179 12585 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40833: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
W20260809 08:51:46.326242 12714 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:46.326416 12714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:46.326433 12714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:46.326465 12714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:46.327571 12714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:46.327610 12714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:46.327623 12714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:46.327646 12714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:46.327668 12714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:46.329126 12714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:40707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:39203,127.10.184.125:42029,127.10.184.124:40833
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:40833
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.184.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:46.329334 12714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:46.329540 12714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 08:51:46.333993 12714 server_base.cc:1034] running on GCE node
W20260809 08:51:46.334010 12721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.333967 12720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:46.333977 12724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:46.334378 12714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:46.334636 12714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:46.335767 12714 hybrid_clock.cc:648] HybridClock initialized: now 1786265506335750 us; error 32 us; skew 500 ppm
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:46.338093 12714 init.cc:377] Logged in from keytab as kudu/127.10.184.124@KRBTEST.COM (short username kudu)
I20260809 08:51:46.338474 12714 webserver.cc:460] Webserver started at http://127.10.184.124:35987/ using document root <none> and password file <none>
I20260809 08:51:46.338682 12714 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:46.338729 12714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:46.338851 12714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:46.339906 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data/instance:
uuid: "da8f8f656ad34631bde742ab6f82dc75"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "a5dc1c52120de3801e02aa919366161c"
server_key_iv: "d1f0ecec2271790b758389b68acd7ee4"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.340247 12714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal/instance:
uuid: "da8f8f656ad34631bde742ab6f82dc75"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "a5dc1c52120de3801e02aa919366161c"
server_key_iv: "d1f0ecec2271790b758389b68acd7ee4"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.341509 12714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 08:51:46.342126 12736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:46.342279 12714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:46.342361 12714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "da8f8f656ad34631bde742ab6f82dc75"
format_stamp: "Formatted at 2026-08-09 08:51:46 on dist-test-slave-ncp9"
server_key: "a5dc1c52120de3801e02aa919366161c"
server_key_iv: "d1f0ecec2271790b758389b68acd7ee4"
server_key_version: "encryptionkey@0"
I20260809 08:51:46.342417 12714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:46.351604 12704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } attempt: 2
I20260809 08:51:46.351886 12714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:46.351945 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.351842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58220)
0809 08:51:46.351892 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.351920 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 08:51:46.352288 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.352208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58236)
0809 08:51:46.352243 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.352267 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 08:51:46.352361 12704 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40833: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40833: connect: Connection refused (error 111)
I20260809 08:51:46.353892 12714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:46.354024 12714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:46.357527 12714 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.124:40833
I20260809 08:51:46.357556 12821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.124:40833 every 8 connection(s)
I20260809 08:51:46.357852 12714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
I20260809 08:51:46.358678 12822 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:46.359392 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 12714
I20260809 08:51:46.359491 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786265241374876-10977-0/minicluster-data/master-2/wal/instance
I20260809 08:51:46.359668 10977 external_mini_cluster.cc:1428] Setting key 8ff636783827c9aa342880bbb94c3c36
I20260809 08:51:46.360391 12822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:46.369302 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.361415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43906 (local address 127.10.184.126:39203)
0809 08:51:46.361486 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.361489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.361683 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:51:46.361752 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.361753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.361763 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.361816 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.362261 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.362968 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.364193 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.364384 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.364440 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.366634 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.366636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.366636 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.366645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.368007 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368413 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.368414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.368415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.368472 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368690 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.368691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.368692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.368803 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.368857 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.369127 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.369215 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:51:46.369529 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.361364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:39203 (local address 127.0.0.1:43906)
0809 08:51:46.361560 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.361603 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.361689 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.361844 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.361846 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.362067 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.362163 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.362168 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.363038 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.363039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.364108 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.364111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.364452 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.364453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.364577 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.365090 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.365100 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.366548 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.368078 (+ 1530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368085 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.368087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.368348 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.368525 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368525 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.368526 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.368628 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.368856 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.368863 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.369013 (+ 150us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.369098 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.369231 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":75,"threads_started":1}
I20260809 08:51:46.369632 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.360121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.10.184.124:40833)
0809 08:51:46.360285 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.360295 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.360347 (+ 52us) server_negotiation.cc:407] Connection header received
0809 08:51:46.360945 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.360953 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.361002 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.361124 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.361833 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.363062 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.364100 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.364720 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.364783 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.366992 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.366999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.367000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.367006 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.368378 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368689 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.368692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.368692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.368746 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.368929 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.368930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.368930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.369055 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.369194 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.369276 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.369531 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":79,"threads_started":1}
I20260809 08:51:46.370155 12822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:46.376125 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.370365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:42029 (local address 127.0.0.1:49916)
0809 08:51:46.370399 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.370408 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.370437 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.370575 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.370576 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.370682 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.370715 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.370716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.371351 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.371352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.371847 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.371849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.372222 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.372223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.372286 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.372699 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.372704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.373785 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.375195 (+ 1410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.375196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.375196 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.375352 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.375568 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.375569 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.375569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.375621 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.375875 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.375876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.375907 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.375959 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.376041 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:51:46.376151 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.370410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49916 (local address 127.10.184.125:42029)
0809 08:51:46.370474 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.370476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.370484 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:46.370505 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.370506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.370515 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.370574 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.370776 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.371279 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.371918 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.372158 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.372215 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.373867 (+ 1652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.373868 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.373869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.373876 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.375100 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.375421 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.375422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.375423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.375501 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.375689 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.375690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.375690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.375819 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.375885 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.376002 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.376071 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 08:51:46.376507 12822 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:46.377084 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.370294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43912 (local address 127.10.184.126:39203)
0809 08:51:46.370328 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.370329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.370336 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:46.370372 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.370373 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.370381 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.370423 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.370914 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.371358 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.371967 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.372138 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.372190 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.374413 (+ 2223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.374414 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.374414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.374420 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.375896 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.376188 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.376189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.376190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.376250 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.376517 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.376519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.376519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.376659 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.376862 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.376921 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.376998 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265505, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:46.382899 12826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.376721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58254)
0809 08:51:46.376774 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.376782 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.376813 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.376949 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.376950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.377078 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.377107 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.377108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.377772 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.377773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.378476 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.378479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.378925 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.378925 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.378958 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.379249 (+ 291us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.379254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.380470 (+ 1216us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.381934 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.381934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.381935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.382105 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.382308 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.382309 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.382309 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.382356 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.382594 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.382595 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.382638 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.382707 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.382791 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 08:51:46.383049 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.376762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.10.184.124:40833)
0809 08:51:46.376793 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.376794 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.376804 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:46.376856 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.376857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.376865 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.376942 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.377194 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.377698 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.378540 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.378860 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.378916 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.380540 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.380541 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.380542 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.380547 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.381850 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.382188 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.382190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.382190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.382247 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.382409 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.382410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.382410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.382527 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.382573 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.382745 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.382851 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":15}
I20260809 08:51:46.384110 12822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: Bootstrap starting.
I20260809 08:51:46.384773 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.377707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.10.184.125:42029)
0809 08:51:46.377759 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.377761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.377789 (+ 28us) server_negotiation.cc:407] Connection header received
0809 08:51:46.377843 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.377844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.377851 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.377886 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.378393 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.378992 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.379831 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.380129 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.380214 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.382455 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.382456 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.382457 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.382462 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.383791 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.384076 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.384077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.384077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.384129 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.384323 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.384324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.384324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.384441 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.384589 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.384625 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.384698 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 08:51:46.385047 12822 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:46.385291 12822 log.cc:824] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:46.386103 12822 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: No bootstrap required, opened a new log
I20260809 08:51:46.386487 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 08:51:46.387590 12822 raft_consensus.cc:348] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.387717 12822 raft_consensus.cc:729] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da8f8f656ad34631bde742ab6f82dc75, State: Initialized, Role: FOLLOWER
I20260809 08:51:46.387794 12822 consensus_queue.cc:260] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.388072 12822 sys_catalog.cc:564] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:46.388043 12828 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.388134 12828 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.387454 12546 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "564d7e5bdbbe4c2ea642596e964c3865"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43912
I20260809 08:51:46.389276 12585 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } attempt: 2
W20260809 08:51:46.390056 12847 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:46.390108 12847 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:46.390257 12714 master_runner.cc:186] Error getting master registration for 127.10.184.126:39203: OK, instance_id { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" instance_seqno: 1786265506044877 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:51:46.392976 12714 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:46.396277 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.389563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58260 (local address 127.10.184.124:40833)
0809 08:51:46.389597 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.389599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.389606 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:46.389666 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.389666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.389674 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.389717 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.390021 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.390630 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.391568 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.391856 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.391917 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.393718 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.393719 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.393720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.393727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.395180 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.395536 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.395537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.395537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.395596 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.395776 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.395777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.395777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.395925 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.395993 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.396125 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.396198 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:46.396281 12590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.389528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58260)
0809 08:51:46.389563 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.389574 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.389619 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.389756 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.389757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.389906 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.389945 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.389947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.390720 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.390721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.391477 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.391480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.391930 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.391931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.392000 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.392395 (+ 395us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.392412 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.393618 (+ 1206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.395261 (+ 1643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.395262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.395262 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.395469 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.395665 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.395666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.395666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.395722 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.395989 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.395989 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.396029 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.396093 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.396185 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 08:51:46.397727 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: Bootstrap starting.
I20260809 08:51:46.398499 12585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:46.398773 12585 log.cc:824] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:46.399565 12585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: No bootstrap required, opened a new log
I20260809 08:51:46.400832 12585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.400980 12585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564d7e5bdbbe4c2ea642596e964c3865, State: Initialized, Role: FOLLOWER
I20260809 08:51:46.401058 12585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.401364 12585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:46.401331 12852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.401423 12546 raft_consensus.cc:482] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:51:46.401472 12852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.401522 12546 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:46.402288 12546 raft_consensus.cc:504] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.402532 12546 leader_election.cc:288] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [CANDIDATE]: Term 1 election: Requested vote from peers f18a90011fac4c169669067ed1baff71 (127.10.184.125:42029), da8f8f656ad34631bde742ab6f82dc75 (127.10.184.124:40833)
W20260809 08:51:46.403105 12872 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:46.403162 12872 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:46.403385 12779 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "564d7e5bdbbe4c2ea642596e964c3865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "da8f8f656ad34631bde742ab6f82dc75"
I20260809 08:51:46.403501 12779 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:46.404354 12779 raft_consensus.cc:2454] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 564d7e5bdbbe4c2ea642596e964c3865 in term 1.
I20260809 08:51:46.404582 12497 leader_election.cc:302] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 564d7e5bdbbe4c2ea642596e964c3865, da8f8f656ad34631bde742ab6f82dc75; no voters:
I20260809 08:51:46.404696 12852 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:46.404844 12852 raft_consensus.cc:686] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [term 1 LEADER]: Becoming Leader. State: Replica: 564d7e5bdbbe4c2ea642596e964c3865, State: Running, Role: LEADER
I20260809 08:51:46.404951 12852 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.405359 12852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564d7e5bdbbe4c2ea642596e964c3865. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.405434 12852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:46.405157 12470 master_runner.cc:186] Error getting master registration for 127.10.184.125:42029: OK, instance_id { permanent_uuid: "f18a90011fac4c169669067ed1baff71" instance_seqno: 1786265506212204 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:51:46.407317 12470 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:51:46.416213 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.409160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58262 (local address 127.10.184.124:40833)
0809 08:51:46.409200 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.409202 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.409782 (+ 580us) server_negotiation.cc:407] Connection header received
0809 08:51:46.409864 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.409865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.409873 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.409900 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.410638 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.411075 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.412070 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.412314 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.412374 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.413865 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.413866 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.413867 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.413873 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.415127 (+ 1254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.415475 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.415476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.415476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.415528 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.415689 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.415690 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.415690 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.415822 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.415896 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.416072 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.416135 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 08:51:46.417006 12882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:46.417218 12882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:46.418983 12779 raft_consensus.cc:1261] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [term 1 FOLLOWER]: Refusing update from remote peer 564d7e5bdbbe4c2ea642596e964c3865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:46.419284 12880 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:46.420496 12828 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564d7e5bdbbe4c2ea642596e964c3865. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.420585 12828 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.425786 12589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.409367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43918 (local address 127.10.184.126:39203)
0809 08:51:46.409424 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.409426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.409802 (+ 376us) server_negotiation.cc:407] Connection header received
0809 08:51:46.409840 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.409841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.409850 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.409897 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.414805 (+ 4908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.415502 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.416491 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.416762 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.416812 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.421866 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.421868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.421868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.421876 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.423678 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.424718 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.424720 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.424720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.424802 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.425155 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.425156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.425157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.425285 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.425363 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.425606 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.425695 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:46.426558 12709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.409566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49942 (local address 127.10.184.125:42029)
0809 08:51:46.409606 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.409607 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.414094 (+ 4487us) server_negotiation.cc:407] Connection header received
0809 08:51:46.414192 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.414194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.414204 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.414233 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.416512 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.417195 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.418457 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.418784 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.418825 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.421888 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.421890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.421890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.421897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.424370 (+ 2473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.424994 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.424995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.424996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.425056 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.425855 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.425857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.425857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.425984 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.426069 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.426380 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.426466 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:46.430588 12852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.430689 12852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:46.431124 12828 sys_catalog.cc:455] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.431221 12828 sys_catalog.cc:458] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.431422 12888 mvcc.cc:205] Tried to move back new op lower bound from 7316543514289520640 to 7316543514235363328. Current Snapshot: MvccSnapshot[applied={T|T < 7316543514289520640}]
I20260809 08:51:46.432726 12882 catalog_manager.cc:1347] Generated new cluster ID: 3c8f8642ff104d148bdbcf139ef9233c
I20260809 08:51:46.432780 12882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:46.443495 12704 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } attempt: 3
Aug 09 08:51:46 dist-test-slave-ncp9 krb5kdc[12455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265506, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:46.452021 12706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.443783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40833 (local address 127.0.0.1:58278)
0809 08:51:46.443834 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:46.443847 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:46.443898 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:46.444030 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:46.444031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:46.444213 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:46.444264 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.444266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.445193 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.445194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.446006 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:46.446009 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.446512 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.446514 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:46.446577 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.447542 (+ 965us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:46.447547 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:46.449118 (+ 1571us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:46.450867 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.450869 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.450869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.451146 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.451358 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:46.451359 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:46.451359 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:46.451423 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:46.451704 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:46.451705 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:46.451750 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 08:51:46.451815 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 08:51:46.451910 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:51:46.452021 12825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:46.443884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.10.184.124:40833)
0809 08:51:46.443921 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.443923 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.443932 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:46.443952 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.443953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.443962 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.444020 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.444328 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.445085 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.446094 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:46.446422 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:46.446503 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:46.449212 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:46.449213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:46.449214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:46.449221 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:46.450778 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.451228 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.451230 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.451230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.451296 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:46.451475 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:46.451476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:46.451476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:46.451637 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:46.451697 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:46.451853 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 08:51:46.451930 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:51:46.453861 12704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: Bootstrap starting.
I20260809 08:51:46.454595 12704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:46.454895 12704 log.cc:824] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:46.456254 12704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: No bootstrap required, opened a new log
I20260809 08:51:46.458271 12704 raft_consensus.cc:348] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.458411 12704 raft_consensus.cc:729] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f18a90011fac4c169669067ed1baff71, State: Initialized, Role: FOLLOWER
I20260809 08:51:46.458492 12704 consensus_queue.cc:260] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } }
I20260809 08:51:46.458770 12897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.458829 12704 sys_catalog.cc:564] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:46.458923 12659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:46.459756 12659 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [term 1 FOLLOWER]: Refusing update from remote peer 564d7e5bdbbe4c2ea642596e964c3865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 08:51:46.460054 12624 tablet.cc:2369] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 08:51:46.460052 12852 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:46.461658 12660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "564d7e5bdbbe4c2ea642596e964c3865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f18a90011fac4c169669067ed1baff71"
I20260809 08:51:46.461892 12897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.461972 12897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564d7e5bdbbe4c2ea642596e964c3865. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.462057 12897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:46.462095 12897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.462137 12897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 08:51:46.462671 12913 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:46.462745 12913 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:46.471374 12915 mvcc.cc:205] Tried to move back new op lower bound from 7316543514289520640 to 7316543514235363328. Current Snapshot: MvccSnapshot[applied={T|T < 7316543514289520640}]
I20260809 08:51:46.472474 12660 raft_consensus.cc:2396] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 564d7e5bdbbe4c2ea642596e964c3865 for term 1 because replica has last-logged OpId of term: 1 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260809 08:51:46.472616 12882 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:46.473078 12880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "564d7e5bdbbe4c2ea642596e964c3865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "564d7e5bdbbe4c2ea642596e964c3865" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 39203 } } peers { permanent_uuid: "f18a90011fac4c169669067ed1baff71" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 42029 } } peers { permanent_uuid: "da8f8f656ad34631bde742ab6f82dc75" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40833 } } }
I20260809 08:51:46.473167 12880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:46.473344 12882 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:46.481765 12882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 564d7e5bdbbe4c2ea642596e964c3865: Generated new TSK 0
I20260809 08:51:46.481936 12882 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 08:51:46.509503 12825 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.507867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.10.184.124:40833)
0809 08:51:46.507910 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.507913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.508823 (+ 910us) server_negotiation.cc:407] Connection header received
0809 08:51:46.508957 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.508959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.508970 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.509001 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.509422 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58280: BlockingRecv error: recv got EOF from 127.0.0.1:58280 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:51:46.510234 12589 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.508234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43926 (local address 127.10.184.126:39203)
0809 08:51:46.508270 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.508272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.509735 (+ 1463us) server_negotiation.cc:407] Connection header received
0809 08:51:46.509831 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.509832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.509842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.509870 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.510175 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43926: BlockingRecv error: recv got EOF from 127.0.0.1:43926 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 08:51:46.510974 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.508564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.10.184.125:42029)
0809 08:51:46.508602 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.508604 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.510459 (+ 1855us) server_negotiation.cc:407] Connection header received
0809 08:51:46.510571 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.510572 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.510583 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.510611 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.510915 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49950: BlockingRecv error: recv got EOF from 127.0.0.1:49950 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:51:46.513353 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.512568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49962 (local address 127.10.184.125:42029)
0809 08:51:46.512613 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.512615 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.512813 (+ 198us) server_negotiation.cc:407] Connection header received
0809 08:51:46.512942 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.512944 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.512961 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.512993 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.513313 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49962: BlockingRecv error: recv got EOF from 127.0.0.1:49962 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:51:46.516158 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.515502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.10.184.125:42029)
0809 08:51:46.515532 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.515534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.515728 (+ 194us) server_negotiation.cc:407] Connection header received
0809 08:51:46.515811 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.515812 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.515820 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.515845 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.516121 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49974: BlockingRecv error: recv got EOF from 127.0.0.1:49974 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:51:46.519999 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.519288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49990 (local address 127.10.184.125:42029)
0809 08:51:46.519318 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.519320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.519533 (+ 213us) server_negotiation.cc:407] Connection header received
0809 08:51:46.519627 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.519627 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.519638 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.519664 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.519961 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49990: BlockingRecv error: recv got EOF from 127.0.0.1:49990 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:51:46.525971 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.525227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50006 (local address 127.10.184.125:42029)
0809 08:51:46.525259 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.525261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.525479 (+ 218us) server_negotiation.cc:407] Connection header received
0809 08:51:46.525591 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.525592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.525602 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.525628 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.525931 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50006: BlockingRecv error: recv got EOF from 127.0.0.1:50006 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:51:46.535985 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.535170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50012 (local address 127.10.184.125:42029)
0809 08:51:46.535207 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.535209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.535435 (+ 226us) server_negotiation.cc:407] Connection header received
0809 08:51:46.535545 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.535546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.535557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.535585 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.535925 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50012: BlockingRecv error: recv got EOF from 127.0.0.1:50012 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:51:46.556236 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.555325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50028 (local address 127.10.184.125:42029)
0809 08:51:46.555367 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.555369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.555654 (+ 285us) server_negotiation.cc:407] Connection header received
0809 08:51:46.555757 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.555758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.555769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.555799 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.556180 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50028: BlockingRecv error: recv got EOF from 127.0.0.1:50028 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:51:46.590214 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.589529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50036 (local address 127.10.184.125:42029)
0809 08:51:46.589583 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.589585 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.589717 (+ 132us) server_negotiation.cc:407] Connection header received
0809 08:51:46.589755 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.589756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.589770 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.589857 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.590153 (+ 296us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50036: BlockingRecv error: recv got EOF from 127.0.0.1:50036 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 08:51:46.655877 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.655275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50052 (local address 127.10.184.125:42029)
0809 08:51:46.655350 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.655351 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.655410 (+ 59us) server_negotiation.cc:407] Connection header received
0809 08:51:46.655458 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.655459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.655468 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.655547 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.655830 (+ 283us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50052: BlockingRecv error: recv got EOF from 127.0.0.1:50052 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 08:51:46.785619 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:46.784991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50064 (local address 127.10.184.125:42029)
0809 08:51:46.785054 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:46.785056 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:46.785138 (+ 82us) server_negotiation.cc:407] Connection header received
0809 08:51:46.785199 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:46.785200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:46.785211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:46.785270 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:46.785560 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50064: BlockingRecv error: recv got EOF from 127.0.0.1:50064 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 08:51:47.043375 12709 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:47.042741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50066 (local address 127.10.184.125:42029)
0809 08:51:47.042812 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.042814 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.042824 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:47.042900 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.042901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.042913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.042962 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:47.043312 (+ 350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50066: BlockingRecv error: recv got EOF from 127.0.0.1:50066 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 08:51:47.390517 12847 catalog_manager.cc:1259] Loaded cluster ID: 3c8f8642ff104d148bdbcf139ef9233c
I20260809 08:51:47.390592 12847 catalog_manager.cc:1552] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: loading cluster ID for follower catalog manager: success
I20260809 08:51:47.391391 12847 catalog_manager.cc:1574] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: acquiring CA information for follower catalog manager: success
I20260809 08:51:47.391690 12847 catalog_manager.cc:1602] T 00000000000000000000000000000000 P da8f8f656ad34631bde742ab6f82dc75: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:51:47.463148 12913 catalog_manager.cc:1259] Loaded cluster ID: 3c8f8642ff104d148bdbcf139ef9233c
I20260809 08:51:47.463220 12913 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: loading cluster ID for follower catalog manager: success
I20260809 08:51:47.464013 12913 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: acquiring CA information for follower catalog manager: success
I20260809 08:51:47.464293 12913 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f18a90011fac4c169669067ed1baff71: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:51:47.559341 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:47.556553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.10.184.125:42029)
0809 08:51:47.556696 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.556698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.556709 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:51:47.556782 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.556783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.556856 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.556948 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:47.557091 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.557733 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.558495 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.558713 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.558770 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:47.558934 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:47.559118 (+ 184us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:47.559189 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:47.559230 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 08:51:47.559253 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":66,"threads_started":1}
I20260809 08:51:47.563066 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:47.561051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.10.184.124:40833)
0809 08:51:47.561160 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.561162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.561171 (+ 9us) server_negotiation.cc:407] Connection header received
0809 08:51:47.561207 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.561208 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.561240 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.561305 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:47.561379 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.561844 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.562509 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.562704 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.562751 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:47.562839 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:47.562927 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:47.562974 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:47.562989 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 08:51:47.563005 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":47,"threads_started":1}
I20260809 08:51:47.567217 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:47.564394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50080 (local address 127.10.184.125:42029)
0809 08:51:47.564439 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.564441 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.564600 (+ 159us) server_negotiation.cc:407] Connection header received
0809 08:51:47.564684 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.564685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.564696 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.564723 (+ 27us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:47.564867 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.565542 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.566452 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.566738 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.566795 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:47.566973 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:47.567002 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:47.567027 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:47.567119 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 08:51:47.567143 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:51:47.567348 13038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:47.564405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43930 (local address 127.10.184.126:39203)
0809 08:51:47.564515 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.564517 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.564617 (+ 100us) server_negotiation.cc:407] Connection header received
0809 08:51:47.564752 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.564753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.564809 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.564885 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:47.565054 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.565611 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.566425 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.566739 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.566805 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:47.566974 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:47.567076 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:47.567130 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:47.567236 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 08:51:47.567263 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":58,"threads_started":1}
I20260809 08:51:47.567430 13028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:47.564736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.10.184.124:40833)
0809 08:51:47.564778 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:47.564780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:47.564860 (+ 80us) server_negotiation.cc:407] Connection header received
0809 08:51:47.564986 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:47.564987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:47.564997 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:47.565039 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:47.565229 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.565900 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.566748 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:47.566999 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:47.567050 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:47.567199 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:47.567228 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:47.567254 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:47.567355 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 08:51:47.567377 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 08:51:47.568928 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12470
I20260809 08:51:47.585311 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12595
I20260809 08:51:47.592682 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 12714
2026-08-09T08:51:47Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2984 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 08:51:47 dist-test-slave-ncp9 krb5kdc[13051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 08:51:47 dist-test-slave-ncp9 krb5kdc[13051](info): set up 2 sockets
Aug 09 08:51:47 dist-test-slave-ncp9 krb5kdc[13051](info): commencing operation
krb5kdc: starting...
W20260809 08:51:48.718830 10977 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/test-user.keytab.
Aug 09 08:51:48 dist-test-slave-ncp9 krb5kdc[13051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T08:51:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T08:51:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.10.184.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
Entry for principal kudu/127.10.184.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab.
I20260809 08:51:48.827348 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44263
--webserver_interface=127.10.184.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 08:51:48.905436 13072 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:48.905622 13072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:48.905637 13072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:48.905656 13072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:48.906780 13072 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:48.906818 13072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:48.906831 13072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:48.906850 13072 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:48.906873 13072 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:48.908296 13072 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.126.keytab
--principal=kudu/127.10.184.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.126:44263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.10.184.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:48.908516 13072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:48.908715 13072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:48.913206 13080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:48.913277 13078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:48.913365 13072 server_base.cc:1034] running on GCE node
W20260809 08:51:48.913213 13077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:48.913717 13072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:48.913985 13072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:48.915118 13072 hybrid_clock.cc:648] HybridClock initialized: now 1786265508915096 us; error 36 us; skew 500 ppm
Aug 09 08:51:48 dist-test-slave-ncp9 krb5kdc[13051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:48.917586 13072 init.cc:377] Logged in from keytab as kudu/127.10.184.126@KRBTEST.COM (short username kudu)
I20260809 08:51:48.918071 13072 webserver.cc:460] Webserver started at http://127.10.184.126:38969/ using document root <none> and password file <none>
I20260809 08:51:48.918319 13072 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:48.918373 13072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:48.918500 13072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:48.919518 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data/instance:
uuid: "3f07d385ddfb4fb7bce9aa6508e64af2"
format_stamp: "Formatted at 2026-08-09 08:51:48 on dist-test-slave-ncp9"
server_key: "f2ae72bf1668767f0c24c77f4b1a10df"
server_key_iv: "05a39ed61137502e0bbce07dafe5dd65"
server_key_version: "encryptionkey@0"
I20260809 08:51:48.919878 13072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance:
uuid: "3f07d385ddfb4fb7bce9aa6508e64af2"
format_stamp: "Formatted at 2026-08-09 08:51:48 on dist-test-slave-ncp9"
server_key: "f2ae72bf1668767f0c24c77f4b1a10df"
server_key_iv: "05a39ed61137502e0bbce07dafe5dd65"
server_key_version: "encryptionkey@0"
I20260809 08:51:48.921167 13072 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 08:51:48.921819 13089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:48.921944 13072 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:48.922013 13072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal
uuid: "3f07d385ddfb4fb7bce9aa6508e64af2"
format_stamp: "Formatted at 2026-08-09 08:51:48 on dist-test-slave-ncp9"
server_key: "f2ae72bf1668767f0c24c77f4b1a10df"
server_key_iv: "05a39ed61137502e0bbce07dafe5dd65"
server_key_version: "encryptionkey@0"
I20260809 08:51:48.922079 13072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:48.935403 13072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:48.937496 13072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:48.937623 13072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:48.941331 13072 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.126:44263
I20260809 08:51:48.941385 13168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.126:44263 every 8 connection(s)
I20260809 08:51:48.941679 13072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/data/info.pb
I20260809 08:51:48.942510 13169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:48.943974 13169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:48 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:48.951251 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13072
I20260809 08:51:48.951362 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-0/wal/instance
I20260809 08:51:48.951561 10977 external_mini_cluster.cc:1428] Setting key d88458953c425c55260eed5561303af5
I20260809 08:51:48.952405 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:48.944595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60672 (local address 127.10.184.126:44263)
0809 08:51:48.944738 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:48.944746 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:48.944806 (+ 60us) server_negotiation.cc:407] Connection header received
0809 08:51:48.945340 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:48.945342 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:48.945359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:48.945439 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:48.945917 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:48.946712 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:48.947498 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:48.947786 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:48.947855 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:48.949723 (+ 1868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:48.949730 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:48.949730 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:48.949736 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:48.950911 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:48.951313 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:48.951316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:48.951316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:48.951420 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:48.951683 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:48.951684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:48.951685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:48.951879 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:48.951940 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:48.952145 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 08:51:48.952261 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
I20260809 08:51:48.952442 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:48.944547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44263 (local address 127.0.0.1:60672)
0809 08:51:48.944706 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:48.944741 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:48.945215 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:48.945451 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:48.945453 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:48.945696 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:48.945850 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:48.945854 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:48.946779 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:48.946780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:48.947438 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:48.947440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:48.947839 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:48.947840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:48.947890 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:48.948305 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:48.948313 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:48.949636 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:48.950985 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:48.950991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:48.950993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:48.951219 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:48.951475 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:48.951476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:48.951476 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:48.951587 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:48.951943 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:48.951950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:48.952022 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 08:51:48.952114 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 08:51:48.952335 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":80,"thread_start_us":61,"threads_started":1}
I20260809 08:51:48.953467 13169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:48.953791 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:48.953684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39662)
0809 08:51:48.953725 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:48.953772 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 08:51:48.953927 13095 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
I20260809 08:51:48.954329 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:48.954240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39674)
0809 08:51:48.954280 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:48.954309 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 08:51:48.954423 13169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:40947: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.10.184.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
Entry for principal kudu/127.10.184.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab.
I20260809 08:51:48.973085 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:40947
--webserver_interface=127.10.184.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:51:49.003639 13169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } attempt: 1
I20260809 08:51:49.004161 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.004024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39690)
0809 08:51:49.004086 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.004129 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 08:51:49.004468 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.004393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39702)
0809 08:51:49.004426 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.004450 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:51:49.004555 13169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:40947: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
W20260809 08:51:49.051476 13179 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:49.051661 13179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:49.051676 13179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:49.051695 13179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:49.052819 13179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:49.052857 13179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:49.052870 13179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:49.052917 13179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:49.052942 13179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:49.054386 13179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.125.keytab
--principal=kudu/127.10.184.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.125:40947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.10.184.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:49.054601 13179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:49.054790 13179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:49.059237 13186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:49.059363 13179 server_base.cc:1034] running on GCE node
W20260809 08:51:49.059302 13185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:49.059329 13192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:49.059721 13179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:49.059979 13179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:49.061112 13179 hybrid_clock.cc:648] HybridClock initialized: now 1786265509061099 us; error 28 us; skew 500 ppm
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:49.063375 13179 init.cc:377] Logged in from keytab as kudu/127.10.184.125@KRBTEST.COM (short username kudu)
I20260809 08:51:49.063758 13179 webserver.cc:460] Webserver started at http://127.10.184.125:36523/ using document root <none> and password file <none>
I20260809 08:51:49.063956 13179 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:49.064002 13179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:49.064121 13179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:49.065124 13179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data/instance:
uuid: "d729ce82be0b41d5bf84ae99f0b7090c"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e5526fda69d209a449194d81238b9c49"
server_key_iv: "f431245ecaacb8e17d86b85ea4485a96"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.065456 13179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal/instance:
uuid: "d729ce82be0b41d5bf84ae99f0b7090c"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e5526fda69d209a449194d81238b9c49"
server_key_iv: "f431245ecaacb8e17d86b85ea4485a96"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.066619 13179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 08:51:49.067235 13206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:49.067379 13179 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 08:51:49.067464 13179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal
uuid: "d729ce82be0b41d5bf84ae99f0b7090c"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e5526fda69d209a449194d81238b9c49"
server_key_iv: "f431245ecaacb8e17d86b85ea4485a96"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.067520 13179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:49.072692 13169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } attempt: 2
I20260809 08:51:49.073060 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.072958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39718)
0809 08:51:49.073001 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.073028 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 08:51:49.073344 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.073272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39730)
0809 08:51:49.073306 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.073328 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:51:49.073416 13169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.125:40947: Network error: Client connection negotiation failed: client connection to 127.10.184.125:40947: connect: Connection refused (error 111)
I20260809 08:51:49.085551 13179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:49.087144 13179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:49.087260 13179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:49.090885 13179 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.125:40947
I20260809 08:51:49.090919 13283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.125:40947 every 8 connection(s)
I20260809 08:51:49.091212 13179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/data/info.pb
I20260809 08:51:49.092036 13284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:49.093513 13284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:49.096952 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13179
I20260809 08:51:49.097051 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-1/wal/instance
I20260809 08:51:49.097225 10977 external_mini_cluster.cc:1428] Setting key cf7845f043f8238e633367ab09a1b663
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:49.101883 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.094118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60674 (local address 127.10.184.126:44263)
0809 08:51:49.094172 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.094175 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.094347 (+ 172us) server_negotiation.cc:407] Connection header received
0809 08:51:49.094809 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.094810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.094824 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.094872 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.095443 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.095914 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.096827 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.097088 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.097150 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.099245 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.099246 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.099247 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.099254 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.100525 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.100986 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.100987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.100987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.101038 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.101306 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.101306 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.101306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.101408 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.101458 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.101766 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.101825 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 08:51:49.102123 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.094086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44263 (local address 127.0.0.1:60674)
0809 08:51:49.094240 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.094275 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.094720 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.094949 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.094953 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.095182 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.095357 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.095362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.095991 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.095993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.096749 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.096751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.097166 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.097167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.097280 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.097746 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.097754 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.099157 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.100607 (+ 1450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.100615 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.100617 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.100918 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.101097 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.101098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.101099 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.101239 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.101465 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.101472 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.101630 (+ 158us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.101755 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.101911 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 08:51:49.102674 13284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.10.184.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.10.184.124@KRBTEST.COM" created.
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:49.111354 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.102872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39742)
0809 08:51:49.102919 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.102930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.102973 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.103177 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.103177 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.103316 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.103369 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.103371 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.104140 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.104141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.105000 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.105004 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.105422 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.105423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.105495 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.105857 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.105862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.108737 (+ 2875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.110359 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.110360 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.110361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.110562 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.110747 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.110748 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.110748 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.110812 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.111033 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.111034 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.111078 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.111150 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.111246 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 08:51:49.111378 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.102930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39742 (local address 127.10.184.125:40947)
0809 08:51:49.103055 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.103058 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.103073 (+ 15us) server_negotiation.cc:407] Connection header received
0809 08:51:49.103091 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.103093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.103103 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.103161 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.103434 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.104059 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.105071 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.105352 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.105400 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.108820 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.108830 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.108831 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.108838 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.110285 (+ 1447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.110637 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.110640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.110640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.110695 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.110864 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.110865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.110865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.110979 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.111033 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.111172 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.111293 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":58,"server-negotiator.queue_time_us":95,"thread_start_us":62,"threads_started":1}
I20260809 08:51:49.112004 13284 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:49.112341 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.112214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45100)
0809 08:51:49.112261 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.112317 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 08:51:49.112473 13208 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
I20260809 08:51:49.112953 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.112816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45106)
0809 08:51:49.112867 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.112931 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 08:51:49.113055 13284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40121: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
Entry for principal kudu/127.10.184.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab.
I20260809 08:51:49.121570 10977 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskAKHc0d/build/release/bin/kudu
/tmp/dist-test-taskAKHc0d/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:40121
--webserver_interface=127.10.184.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.10.184.84:36473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 08:51:49.162294 13284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } attempt: 1
I20260809 08:51:49.162786 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.162655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45112)
0809 08:51:49.162714 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.162756 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:51:49.163110 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.163041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45122)
0809 08:51:49.163074 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.163096 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260809 08:51:49.163211 13284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40121: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
I20260809 08:51:49.164533 13169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } attempt: 3
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:49.170928 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.164849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39748 (local address 127.10.184.125:40947)
0809 08:51:49.164887 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.164906 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.164917 (+ 11us) server_negotiation.cc:407] Connection header received
0809 08:51:49.164981 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.164982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.164994 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.165040 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.165358 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.165846 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.166527 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.166742 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.166797 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.168621 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.168622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.168623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.168630 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.169929 (+ 1299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.170257 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.170258 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.170259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.170308 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.170470 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.170471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.170471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.170573 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.170629 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.170765 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.170843 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:49.170940 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.164802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39748)
0809 08:51:49.164850 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.164860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.164923 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.165054 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.165055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.165220 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.165285 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.165287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.165922 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.165923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.166455 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.166457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.166805 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.166806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.166892 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.167330 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.167333 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.168530 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.170010 (+ 1480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.170011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.170011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.170198 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.170362 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.170363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.170363 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.170421 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.170628 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.170629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.170677 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.170738 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.170851 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:51:49.171286 13169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:49.171581 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.171482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45132)
0809 08:51:49.171521 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.171557 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:51:49.171911 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.171794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45146)
0809 08:51:49.171843 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.171881 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 08:51:49.172003 13169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40121: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
W20260809 08:51:49.201392 13295 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 08:51:49.201575 13295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 08:51:49.201591 13295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 08:51:49.201608 13295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 08:51:49.202699 13295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 08:51:49.202732 13295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 08:51:49.202744 13295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 08:51:49.202755 13295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 08:51:49.202771 13295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 08:51:49.203152 13169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } attempt: 1
I20260809 08:51:49.203526 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.203401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45148)
0809 08:51:49.203460 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.203498 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 08:51:49.203845 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.203762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45162)
0809 08:51:49.203797 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.203824 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 08:51:49.203949 13169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40121: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
I20260809 08:51:49.204456 13295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.10.184.84:36473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.10.184.126:44263,127.10.184.125:40947,127.10.184.124:40121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/krb5kdc/kudu_127.10.184.124.keytab
--principal=kudu/127.10.184.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.10.184.124:40121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.10.184.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 07:55:31 UTC on ccycloud.quasar-tljzob.root.comops.site
build id 4575936
I20260809 08:51:49.204694 13295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 08:51:49.204928 13295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 08:51:49.209386 13304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:49.209518 13307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 08:51:49.209393 13302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 08:51:49.209635 13295 server_base.cc:1034] running on GCE node
I20260809 08:51:49.209882 13295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 08:51:49.210139 13295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 08:51:49.211272 13295 hybrid_clock.cc:648] HybridClock initialized: now 1786265509211258 us; error 28 us; skew 500 ppm
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 08:51:49.213620 13295 init.cc:377] Logged in from keytab as kudu/127.10.184.124@KRBTEST.COM (short username kudu)
I20260809 08:51:49.213979 13295 webserver.cc:460] Webserver started at http://127.10.184.124:41393/ using document root <none> and password file <none>
I20260809 08:51:49.214187 13295 fs_manager.cc:359] Metadata directory not provided
I20260809 08:51:49.214231 13295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 08:51:49.214313 13295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 08:51:49.215240 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data/instance:
uuid: "5da276a4d2cc4d8296c3c2e65e96dcba"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e51ef91a6d3a92cfa5c9cf1be173184a"
server_key_iv: "fe510af453fee9aba4667f4fc691784f"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.215519 13295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal/instance:
uuid: "5da276a4d2cc4d8296c3c2e65e96dcba"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e51ef91a6d3a92cfa5c9cf1be173184a"
server_key_iv: "fe510af453fee9aba4667f4fc691784f"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.216676 13295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 08:51:49.217391 13316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:49.217536 13295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 08:51:49.217620 13295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data,/tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal
uuid: "5da276a4d2cc4d8296c3c2e65e96dcba"
format_stamp: "Formatted at 2026-08-09 08:51:49 on dist-test-slave-ncp9"
server_key: "e51ef91a6d3a92cfa5c9cf1be173184a"
server_key_iv: "fe510af453fee9aba4667f4fc691784f"
server_key_version: "encryptionkey@0"
I20260809 08:51:49.217676 13295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 08:51:49.231359 13284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } attempt: 2
I20260809 08:51:49.231756 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.231614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45170)
0809 08:51:49.231677 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.231719 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 08:51:49.232061 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.231983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45176)
0809 08:51:49.232022 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.232045 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260809 08:51:49.232141 13284 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.10.184.124:40121: Network error: Client connection negotiation failed: client connection to 127.10.184.124:40121: connect: Connection refused (error 111)
I20260809 08:51:49.244767 13295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 08:51:49.246537 13295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 08:51:49.246661 13295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 08:51:49.250252 13295 rpc_server.cc:307] RPC server started. Bound to: 127.10.184.124:40121
I20260809 08:51:49.250303 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.10.184.124:40121 every 8 connection(s)
I20260809 08:51:49.250612 13295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/data/info.pb
I20260809 08:51:49.251464 13410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 08:51:49.252942 13410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:49.255861 10977 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskAKHc0d/build/release/bin/kudu as pid 13295
I20260809 08:51:49.255977 10977 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskAKHc0d/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786265241374876-10977-0/minicluster-data/master-2/wal/instance
I20260809 08:51:49.256162 10977 external_mini_cluster.cc:1428] Setting key cf34d3304710b8e58fe3e531cb593260
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
I20260809 08:51:49.261431 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.253562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60678 (local address 127.10.184.126:44263)
0809 08:51:49.253628 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.253630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.253783 (+ 153us) server_negotiation.cc:407] Connection header received
0809 08:51:49.254259 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.254260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.254273 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.254323 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.254888 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.255350 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.256374 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.256635 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.256711 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.258846 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.258848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.258848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.258856 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.260105 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.260528 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.260528 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.260529 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.260574 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.260804 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.260805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.260805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.260934 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.260992 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.261257 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.261339 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:51:49.261466 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.253535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.126:44263 (local address 127.0.0.1:60678)
0809 08:51:49.253674 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.253707 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.254160 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.254385 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.254388 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.254611 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.254799 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.254803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.255426 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.255427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.256293 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.256296 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.256708 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.256709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.256862 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.257333 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.257342 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.258766 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.260177 (+ 1411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.260185 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.260187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.260460 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.260613 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.260614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.260614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.260748 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.260989 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.260995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.261134 (+ 139us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.261219 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.261323 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:49.261972 13410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } has no permanent_uuid. Determining permanent_uuid...
I20260809 08:51:49.264820 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.256517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45188 (local address 127.10.184.124:40121)
0809 08:51:49.256680 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.256685 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.256736 (+ 51us) server_negotiation.cc:407] Connection header received
0809 08:51:49.256797 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.256800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.256820 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.256887 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.257456 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.258288 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.259130 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.259549 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.259614 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.262032 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.262040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.262041 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.262047 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.263627 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.263928 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.263931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.263931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.263993 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.264173 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.264174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.264174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.264327 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.264456 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.264639 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.264727 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:49.268256 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.262203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.125:40947 (local address 127.0.0.1:39750)
0809 08:51:49.262240 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.262248 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.262280 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.262463 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.262464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.262620 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.262669 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.262671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.263484 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.263484 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.263935 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.263937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.264342 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.264343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.264396 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.264726 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.264731 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.265967 (+ 1236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.267362 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.267363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.267364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.267560 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.267732 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.267732 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.267733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.267783 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.268004 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.268004 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.268042 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.268104 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.268196 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:51:49.268287 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.262268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39750 (local address 127.10.184.125:40947)
0809 08:51:49.262317 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.262319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.262327 (+ 8us) server_negotiation.cc:407] Connection header received
0809 08:51:49.262393 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.262394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.262405 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.262459 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.262740 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.263410 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.263993 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.264277 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.264342 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.266041 (+ 1699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.266042 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.266043 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.266048 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.267293 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.267627 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.267627 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.267627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.267678 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.267829 (+ 151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.267829 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.267830 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.267946 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.268010 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.268123 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.268201 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 08:51:49.268575 13410 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.126@KRBTEST.COM
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.124@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:49.273234 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.265611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.10.184.126:44263)
0809 08:51:49.265649 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.265651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.265733 (+ 82us) server_negotiation.cc:407] Connection header received
0809 08:51:49.265791 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.265792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.265801 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.265837 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.266305 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.266963 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.267689 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.267946 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.268005 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.270497 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.270499 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.270499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.270506 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.271835 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.272141 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.272142 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.272142 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.272208 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.272373 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.272374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.272374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.272478 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.273032 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.273068 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.273141 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 08:51:49.274940 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.268777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45194)
0809 08:51:49.268821 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.268831 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.268873 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.269021 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.269022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.269160 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.269193 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.269194 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.269799 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.269800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.270516 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.270519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.270922 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.270923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.270965 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.271264 (+ 299us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.271268 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.272300 (+ 1032us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.274016 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.274017 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.274017 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.274169 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.274354 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.274355 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.274355 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.274397 (+ 42us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.274633 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.274634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.274675 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.274747 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.274836 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260809 08:51:49.274953 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.268817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45194 (local address 127.10.184.124:40121)
0809 08:51:49.268847 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.268848 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.268858 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:49.268963 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.268964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.268972 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.269015 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.269252 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.269732 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.270581 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.270850 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.270912 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.272374 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.272376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.272376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.272382 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.273944 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.274232 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.274233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.274233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.274292 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.274446 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.274446 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.274447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.274566 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.274616 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.274766 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.274880 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":15}
I20260809 08:51:49.276350 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: Bootstrap starting.
I20260809 08:51:49.277388 13410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:49.277688 13410 log.cc:824] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: Log is configured to *not* fsync() on all Append() calls
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.10.184.125@KRBTEST.COM
I20260809 08:51:49.278366 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: No bootstrap required, opened a new log
I20260809 08:51:49.279100 13169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } attempt: 2
I20260809 08:51:49.280387 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.280525 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5da276a4d2cc4d8296c3c2e65e96dcba, State: Initialized, Role: FOLLOWER
I20260809 08:51:49.280635 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.273831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.10.184.125:40947)
0809 08:51:49.273867 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.273868 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.273908 (+ 40us) server_negotiation.cc:407] Connection header received
0809 08:51:49.273950 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.273951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.273960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.274005 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.274521 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.275149 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.275948 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.276123 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.276172 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.278363 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.278364 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.278364 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.278371 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.279728 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.279997 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.279998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.279998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.280047 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.280211 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.280212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.280212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.280305 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.280457 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.280491 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.280553 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 08:51:49.280612 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.280958 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:49.281069 13418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.281410 13418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.281916 10977 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265508, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.126@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
W20260809 08:51:49.283437 13437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:49.283543 13437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:49.282814 13134 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60690
I20260809 08:51:49.283134 13295 master_runner.cc:186] Error getting master registration for 127.10.184.126:44263: OK, instance_id { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" instance_seqno: 1786265508940272 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:51:49.285720 13295 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:51:49.286258 13173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.279327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45196)
0809 08:51:49.279371 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.279382 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.279425 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.279548 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.279549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.279708 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.279747 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.279749 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.280673 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.280674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.281398 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.281401 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.281864 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.281865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.281924 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.282328 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.282333 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.283586 (+ 1253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.285248 (+ 1662us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.285249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.285250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.285451 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.285641 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.285642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.285642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.285720 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.285946 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.285947 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.286009 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.286071 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.286172 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 08:51:49.286262 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.279374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45196 (local address 127.10.184.124:40121)
0809 08:51:49.279429 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.279431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.279441 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:49.279475 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.279476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.279484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.279534 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.279831 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.280471 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.281484 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.281751 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.281812 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.283689 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.283690 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.283690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.283696 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.285155 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.285525 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.285526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.285526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.285585 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.285774 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.285774 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.285775 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.285897 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.285950 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.286107 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.286181 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 08:51:49.287603 13169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: Bootstrap starting.
I20260809 08:51:49.288393 13169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:49.288695 13169 log.cc:824] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:49.289573 13169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: No bootstrap required, opened a new log
I20260809 08:51:49.290941 13169 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.291086 13169 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f07d385ddfb4fb7bce9aa6508e64af2, State: Initialized, Role: FOLLOWER
I20260809 08:51:49.291170 13169 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.291456 13169 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:49.291427 13440 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.291512 13440 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.291567 13134 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 08:51:49.291615 13134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:49.292225 13134 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.292414 13134 leader_election.cc:288] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [CANDIDATE]: Term 1 election: Requested vote from peers d729ce82be0b41d5bf84ae99f0b7090c (127.10.184.125:40947), 5da276a4d2cc4d8296c3c2e65e96dcba (127.10.184.124:40121)
I20260809 08:51:49.292922 13364 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba"
I20260809 08:51:49.293037 13364 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:49.294108 13364 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f07d385ddfb4fb7bce9aa6508e64af2 in term 1.
I20260809 08:51:49.294394 13091 leader_election.cc:302] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f07d385ddfb4fb7bce9aa6508e64af2, 5da276a4d2cc4d8296c3c2e65e96dcba; no voters:
I20260809 08:51:49.294519 13440 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 08:51:49.294646 13440 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [term 1 LEADER]: Becoming Leader. State: Replica: 3f07d385ddfb4fb7bce9aa6508e64af2, State: Running, Role: LEADER
I20260809 08:51:49.294744 13440 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
W20260809 08:51:49.295034 13458 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:49.295087 13458 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:49.295186 13467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f07d385ddfb4fb7bce9aa6508e64af2. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.295274 13467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:49.295308 13072 master_runner.cc:186] Error getting master registration for 127.10.184.125:40947: OK, instance_id { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" instance_seqno: 1786265509089902 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 08:51:49.296798 13072 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 08:51:49.304383 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.294241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45208 (local address 127.10.184.124:40121)
0809 08:51:49.294301 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.294302 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.296998 (+ 2696us) server_negotiation.cc:407] Connection header received
0809 08:51:49.297079 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.297080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.297091 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.297117 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.297789 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.298499 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.299830 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.300096 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.300138 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.301642 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.301643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.301644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.301651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.303196 (+ 1545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.303570 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.303571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.303572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.303625 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.303840 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.303841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.303841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.303957 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.304021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.304222 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.304299 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 08:51:49.304680 13469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 08:51:49.304946 13469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 08:51:49.306478 13174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.293648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60704 (local address 127.10.184.126:44263)
0809 08:51:49.293682 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.293684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.297022 (+ 3338us) server_negotiation.cc:407] Connection header received
0809 08:51:49.297044 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.297045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.297062 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.297124 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.298135 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.298783 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.299944 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.300266 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.300303 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.303233 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.303234 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.303235 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.303241 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.304841 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.305444 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.305445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.305445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.305513 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.305803 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.305804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.305804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.305937 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.306011 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.306316 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.306390 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:51:49.309756 13364 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [term 1 FOLLOWER]: Refusing update from remote peer 3f07d385ddfb4fb7bce9aa6508e64af2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:49.310063 13467 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 08:51:49.311262 13418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f07d385ddfb4fb7bce9aa6508e64af2. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.311352 13418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.312860 13467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.312925 13289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.297673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.10.184.125:40947)
0809 08:51:49.297713 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.297714 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.300494 (+ 2780us) server_negotiation.cc:407] Connection header received
0809 08:51:49.300611 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.300612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.300622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.300649 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.305127 (+ 4478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.305791 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.306748 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.307045 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.307085 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.308573 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.308574 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.308574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.308581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.310162 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.311711 (+ 1549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.311713 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.311713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.311771 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.311960 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.311961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.311961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.312113 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.312201 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.312733 (+ 532us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.312813 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:51:49.312956 13467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: This master's current role is: LEADER
I20260809 08:51:49.313426 13418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.313508 13418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.313709 13474 mvcc.cc:205] Tried to move back new op lower bound from 7316543526130376704 to 7316543526072008704. Current Snapshot: MvccSnapshot[applied={T|T < 7316543526130376704}]
I20260809 08:51:49.313814 13475 mvcc.cc:205] Tried to move back new op lower bound from 7316543526130376704 to 7316543526072008704. Current Snapshot: MvccSnapshot[applied={T|T < 7316543526130376704}]
I20260809 08:51:49.321265 13469 catalog_manager.cc:1347] Generated new cluster ID: 476a88614dc0435b81c51e6cb7d7a7b4
I20260809 08:51:49.321321 13469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 08:51:49.323271 13284 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } attempt: 3
Aug 09 08:51:49 dist-test-slave-ncp9 krb5kdc[13051](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786265509, etypes {rep=17 tkt=17 ses=17}, kudu/127.10.184.125@KRBTEST.COM for kudu/127.10.184.124@KRBTEST.COM
I20260809 08:51:49.330935 13469 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 08:51:49.331001 13286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.323510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.10.184.124:40121 (local address 127.0.0.1:45222)
0809 08:51:49.323560 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 08:51:49.323570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 08:51:49.323610 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 08:51:49.323841 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 08:51:49.323842 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 08:51:49.323985 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 08:51:49.324018 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.324019 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.324858 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.324859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.325642 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 08:51:49.325644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.326135 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.326135 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 08:51:49.326200 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.326580 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 08:51:49.326583 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 08:51:49.327929 (+ 1346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 08:51:49.329628 (+ 1699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.329629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.329630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.329854 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.330301 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 08:51:49.330302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 08:51:49.330303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 08:51:49.330380 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 08:51:49.330707 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 08:51:49.330707 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 08:51:49.330761 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 08:51:49.330819 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 08:51:49.330907 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 08:51:49.331456 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:49.323707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45222 (local address 127.10.184.124:40121)
0809 08:51:49.323740 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.323741 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.323747 (+ 6us) server_negotiation.cc:407] Connection header received
0809 08:51:49.323764 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.323764 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.323773 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.323809 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.324075 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.324785 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.325736 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:49.326048 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:49.326121 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:49.328058 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 08:51:49.328059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 08:51:49.328059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 08:51:49.328066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 08:51:49.329546 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.330169 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.330170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.330170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.330228 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 08:51:49.330466 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 08:51:49.330467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 08:51:49.330467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 08:51:49.330607 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 08:51:49.330663 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:49.331304 (+ 641us) server_negotiation.cc:299] Negotiation successful
0809 08:51:49.331374 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 08:51:49.332127 13469 catalog_manager.cc:1504] Loading token signing keys...
I20260809 08:51:49.333024 13284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: Bootstrap starting.
I20260809 08:51:49.333735 13284 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: Neither blocks nor log segments found. Creating new log.
I20260809 08:51:49.334012 13284 log.cc:824] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: Log is configured to *not* fsync() on all Append() calls
I20260809 08:51:49.337819 13469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2: Generated new TSK 0
I20260809 08:51:49.337975 13469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 08:51:49.338531 13284 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: No bootstrap required, opened a new log
I20260809 08:51:49.340493 13284 raft_consensus.cc:348] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.340629 13284 raft_consensus.cc:729] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d729ce82be0b41d5bf84ae99f0b7090c, State: Initialized, Role: FOLLOWER
I20260809 08:51:49.340724 13284 consensus_queue.cc:260] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } }
I20260809 08:51:49.341058 13284 sys_catalog.cc:564] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 08:51:49.341089 13245 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d729ce82be0b41d5bf84ae99f0b7090c"
I20260809 08:51:49.341205 13245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [term 0 FOLLOWER]: Advancing to term 1
I20260809 08:51:49.341965 13245 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f07d385ddfb4fb7bce9aa6508e64af2 in term 1.
I20260809 08:51:49.342157 13244 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [term 1 FOLLOWER]: Refusing update from remote peer 3f07d385ddfb4fb7bce9aa6508e64af2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 08:51:49.342553 13440 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 08:51:49.342902 13505 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 08:51:49.342954 13505 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 08:51:49.344375 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.344506 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.344594 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f07d385ddfb4fb7bce9aa6508e64af2. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.344691 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.344758 13482 sys_catalog.cc:455] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.344843 13482 sys_catalog.cc:458] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c [sys.catalog]: This master's current role is: FOLLOWER
I20260809 08:51:49.353439 13511 mvcc.cc:205] Tried to move back new op lower bound from 7316543526243160064 to 7316543526072008704. Current Snapshot: MvccSnapshot[applied={T|T < 7316543526130376704}]
W20260809 08:51:49.364001 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.362509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.10.184.125:40947)
0809 08:51:49.362558 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.362560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.363394 (+ 834us) server_negotiation.cc:407] Connection header received
0809 08:51:49.363493 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.363494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.363505 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.363533 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.363936 (+ 403us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39768: BlockingRecv error: recv got EOF from 127.0.0.1:39768 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 08:51:49.364816 13174 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.362892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60714 (local address 127.10.184.126:44263)
0809 08:51:49.362925 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.362927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.364333 (+ 1406us) server_negotiation.cc:407] Connection header received
0809 08:51:49.364425 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.364426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.364435 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.364461 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.364751 (+ 290us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60714: BlockingRecv error: recv got EOF from 127.0.0.1:60714 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:51:49.365622 13415 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.363165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45224 (local address 127.10.184.124:40121)
0809 08:51:49.363197 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.363199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.365138 (+ 1939us) server_negotiation.cc:407] Connection header received
0809 08:51:49.365254 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.365255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.365264 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.365291 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.365565 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45224: BlockingRecv error: recv got EOF from 127.0.0.1:45224 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:51:49.367810 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.367221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39780 (local address 127.10.184.125:40947)
0809 08:51:49.367263 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.367265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.367346 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:51:49.367441 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.367442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.367457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.367482 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.367773 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39780: BlockingRecv error: recv got EOF from 127.0.0.1:39780 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260809 08:51:49.370913 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.370145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.10.184.125:40947)
0809 08:51:49.370189 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.370190 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.370417 (+ 227us) server_negotiation.cc:407] Connection header received
0809 08:51:49.370508 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.370509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.370518 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.370545 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.370844 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39792: BlockingRecv error: recv got EOF from 127.0.0.1:39792 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 08:51:49.374688 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.374006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39794 (local address 127.10.184.125:40947)
0809 08:51:49.374036 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.374037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.374259 (+ 222us) server_negotiation.cc:407] Connection header received
0809 08:51:49.374346 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.374347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.374355 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.374380 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.374653 (+ 273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39794: BlockingRecv error: recv got EOF from 127.0.0.1:39794 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260809 08:51:49.380467 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.379724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.10.184.125:40947)
0809 08:51:49.379756 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.379757 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.379988 (+ 231us) server_negotiation.cc:407] Connection header received
0809 08:51:49.380094 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.380095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.380105 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.380132 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.380431 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39804: BlockingRecv error: recv got EOF from 127.0.0.1:39804 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260809 08:51:49.390359 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.389581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39818 (local address 127.10.184.125:40947)
0809 08:51:49.389625 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.389626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.389862 (+ 236us) server_negotiation.cc:407] Connection header received
0809 08:51:49.389956 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.389957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.389967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.389994 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.390318 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39818: BlockingRecv error: recv got EOF from 127.0.0.1:39818 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 08:51:49.408383 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.407554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39828 (local address 127.10.184.125:40947)
0809 08:51:49.407591 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.407593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.407849 (+ 256us) server_negotiation.cc:407] Connection header received
0809 08:51:49.407947 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.407947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.407958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.407985 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.408336 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39828: BlockingRecv error: recv got EOF from 127.0.0.1:39828 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 08:51:49.442351 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.441685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.10.184.125:40947)
0809 08:51:49.441740 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.441743 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.441833 (+ 90us) server_negotiation.cc:407] Connection header received
0809 08:51:49.441894 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.441895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.441907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.441956 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.442290 (+ 334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39838: BlockingRecv error: recv got EOF from 127.0.0.1:39838 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 08:51:49.508109 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.507397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39840 (local address 127.10.184.125:40947)
0809 08:51:49.507442 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.507444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.507525 (+ 81us) server_negotiation.cc:407] Connection header received
0809 08:51:49.507605 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.507606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.507615 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.507664 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.508035 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39840: BlockingRecv error: recv got EOF from 127.0.0.1:39840 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 08:51:49.637893 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.637219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39852 (local address 127.10.184.125:40947)
0809 08:51:49.637288 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.637290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.637365 (+ 75us) server_negotiation.cc:407] Connection header received
0809 08:51:49.637449 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.637450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.637462 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.637503 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.637829 (+ 326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39852: BlockingRecv error: recv got EOF from 127.0.0.1:39852 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 08:51:49.878404 13590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f07d385ddfb4fb7bce9aa6508e64af2" member_type: VOTER last_known_addr { host: "127.10.184.126" port: 44263 } } peers { permanent_uuid: "d729ce82be0b41d5bf84ae99f0b7090c" member_type: VOTER last_known_addr { host: "127.10.184.125" port: 40947 } } peers { permanent_uuid: "5da276a4d2cc4d8296c3c2e65e96dcba" member_type: VOTER last_known_addr { host: "127.10.184.124" port: 40121 } } }
I20260809 08:51:49.878551 13590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f07d385ddfb4fb7bce9aa6508e64af2 [sys.catalog]: This master's current role is: LEADER
W20260809 08:51:49.895534 13289 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 08:51:49.894928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39860 (local address 127.10.184.125:40947)
0809 08:51:49.895004 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:49.895006 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:49.895016 (+ 10us) server_negotiation.cc:407] Connection header received
0809 08:51:49.895064 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:49.895065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:49.895077 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:49.895126 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 08:51:49.895471 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39860: BlockingRecv error: recv got EOF from 127.0.0.1:39860 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 08:51:50.284006 13437 catalog_manager.cc:1259] Loaded cluster ID: 476a88614dc0435b81c51e6cb7d7a7b4
I20260809 08:51:50.284075 13437 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: loading cluster ID for follower catalog manager: success
I20260809 08:51:50.284869 13437 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: acquiring CA information for follower catalog manager: success
I20260809 08:51:50.285216 13437 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5da276a4d2cc4d8296c3c2e65e96dcba: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:51:50.343351 13505 catalog_manager.cc:1259] Loaded cluster ID: 476a88614dc0435b81c51e6cb7d7a7b4
I20260809 08:51:50.343420 13505 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: loading cluster ID for follower catalog manager: success
I20260809 08:51:50.344220 13505 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: acquiring CA information for follower catalog manager: success
I20260809 08:51:50.344529 13505 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d729ce82be0b41d5bf84ae99f0b7090c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 08:51:50.416801 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:50.413954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39868 (local address 127.10.184.125:40947)
0809 08:51:50.414092 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:50.414094 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:50.414193 (+ 99us) server_negotiation.cc:407] Connection header received
0809 08:51:50.414272 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:50.414273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:50.414344 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:50.414417 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:50.414589 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.415232 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.416042 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.416248 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.416303 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:50.416466 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:50.416578 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:50.416644 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:50.416694 (+ 50us) server_negotiation.cc:299] Negotiation successful
0809 08:51:50.416714 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":73,"threads_started":1}
I20260809 08:51:50.421322 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:50.418605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.10.184.124:40121)
0809 08:51:50.418728 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:50.418730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:50.418737 (+ 7us) server_negotiation.cc:407] Connection header received
0809 08:51:50.418770 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:50.418771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:50.418805 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:50.418868 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:50.418993 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.419518 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.420578 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.420779 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.420846 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:50.421000 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:50.421090 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:50.421165 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:50.421222 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 08:51:50.421243 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":69,"threads_started":1}
I20260809 08:51:50.425840 13609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:50.422701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39884 (local address 127.10.184.125:40947)
0809 08:51:50.422785 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:50.422787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:50.422922 (+ 135us) server_negotiation.cc:407] Connection header received
0809 08:51:50.422977 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:50.422978 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:50.422989 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:50.423033 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:50.423180 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.423892 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425034 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.425321 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425358 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:50.425539 (+ 181us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:50.425570 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:50.425595 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:50.425744 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 08:51:50.425765 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 08:51:50.426321 13622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:50.423040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.10.184.124:40121)
0809 08:51:50.423071 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:50.423073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:50.423282 (+ 209us) server_negotiation.cc:407] Connection header received
0809 08:51:50.423310 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:50.423311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:50.423321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:50.423368 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:50.423510 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.424205 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425405 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.425698 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425752 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:50.425964 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:50.425994 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:50.426032 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:50.426236 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 08:51:50.426256 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 08:51:50.426692 13629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 08:51:50.422706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60724 (local address 127.10.184.126:44263)
0809 08:51:50.422817 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 08:51:50.422819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 08:51:50.422970 (+ 151us) server_negotiation.cc:407] Connection header received
0809 08:51:50.423106 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 08:51:50.423107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 08:51:50.423171 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 08:51:50.423224 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 08:51:50.423424 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.424204 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425412 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 08:51:50.425708 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 08:51:50.425744 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 08:51:50.426176 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 08:51:50.426286 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 08:51:50.426323 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 08:51:50.426590 (+ 267us) server_negotiation.cc:299] Negotiation successful
0809 08:51:50.426615 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":52,"threads_started":1}
I20260809 08:51:50.433916 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13072
I20260809 08:51:50.448565 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13179
I20260809 08:51:50.467104 10977 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskAKHc0d/build/release/bin/kudu with pid 13295
2026-08-09T08:51:50Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2876 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5860 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (269083 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST